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Further  Actions  Needed  to  Address  Challenges  and 
Improve  Accountability 


Why  GAO  Did  This  Study 

GAO  designated  DOD’s  multibillion 
dollar  business  systems  modernization 
program  as  high  risk  in  1995,  and, 
since  then,  has  provided  a  series  of 
recommendations  aimed  at 
strengthening  DOD’s  institutional 
approach  to  modernization  and 
reducing  the  risk  associated  with  key 
investments.  The  act  requires  the 
department  to  report  on  actions  taken 
relative  to  its  business  systems 
modernization  efforts  and  GAO  to 
assess  DOD’s  actions  to  comply  with 
the  act.  In  evaluating  DOD’s 
compliance,  GAO  analyzed,  for 
example,  the  latest  version  of  the 
business  enterprise  architecture  and 
enterprise  transition  plan,  investment 
management  policies  and  procedures, 
and  certification  actions  for  its  business 
system  investments. 

What  GAO  Recommends 

GAO  is  making  recommendations  to 
help  ensure  that  the  department’s 
modernization  program  is  fully 
compliant  with  provisions  of  the  act 
and  to  improve  the  department’s 
architecture,  transition  plan,  and 
business  system  investment 
management  and  human  capital 
management  within  the  Office  of  the 
Deputy  Chief  Management  Officer. 
DOD  concurred  with  two 
recommendations,  partially  concurred 
with  three,  and  did  not  concur  with 
three.  GAO  continues  to  believe  its 
recommendations  are  warranted  given 
the  department’s  need  to  more 
effectively  manage  its  billions  of  dollars 
of  business  system  investments  and 
minimize  or  eliminate  system  overlap 
and  duplication  as  appropriate. 
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contact  Valerie  C.  Melvin  at  (202)  512-6304  or 
melvinv@gao.gov. 


What  GAO  Found 

The  Department  of  Defense  (DOD)  continues  efforts  to  establish  a  business 
enterprise  architecture  (a  modernization  blueprint)  and  transition  plan  and 
modernize  its  business  systems  and  processes,  in  compliance  with  key 
provisions  of  the  Ronald  W.  Reagan  National  Defense  Authorization  Act  for 
Fiscal  Year  2005  and  amendments.  Nonetheless,  long-standing  challenges 
remain.  The  following  table  reflects  the  status  of  DOD’s  actions  to  fulfill  selected 
requirements  of  the  act. 


Act’s  requirements 

Status  of  DOD  actions 

Develop  a  business 
enterprise  architecture 

DOD  continues  to  develop  content  for  its  business  enterprise 
architecture,  such  as  business  rules,  and  is  proceeding  with  efforts  to 
extend  the  architecture  to  its  components.  However,  even  though 

DOD  has  spent  more  than  10  years  and  at  least  $379  million  on  its 
business  enterprise  architecture,  its  ability  to  use  the  architecture  to 
guide  and  constrain  investments  has  been  limited  by,  among  other 
things,  the  lack  of  a  detailed  plan. 

Develop  an  enterprise 
transition  plan 

The  department’s  latest  version  of  its  transition  plan  included  data  on 
more  than  1 ,200  covered  defense  business  systems;  however, 
important  content,  such  as  time-phased  milestones  and  performance 
measures,  is  still  needed  to  address  the  act’s  requirements. 

Establish  an  investment 
approval  and  accountability 
structure  along  with  an 
investment  review  process 

DOD  has  taken  steps  to  establish  a  portfolio-based  approach  to 
certifying  defense  business  systems,  including  the  establishment  of  a 
corporate-level  board  to  oversee  the  approach  and  guidance  for 
selecting,  controlling,  and  evaluating  the  investment  portfolio. 

However,  it  has  yet  to  fully  establish  the  foundation  for  its  new 
portfolio-level  investment  management  process  or  the  criteria  and 
procedures  for  making  portfolio-based  investment  decisions. 

Certify  any  business  system 
program  costing  in  excess 
of  $1  million  as  compliant 
with  the  business  enterprise 
architecture  and  as  having 
undertaken  appropriate 
business  process 
reengineering 

DOD’s  portfolio-based  investment  approach  included  reviewing  and 
certifying  more  than  1 ,200  business  systems  for  fiscal  year  2013, 
totaling  about  $6.8  billion  in  funding.  However,  while  DOD  continues 
to  perform  compliance  assertions,  it  has  not  ensured  the  accuracy  of 
business  enterprise  architecture  alignment  through  validation  of 
individual  investments.  Further,  appropriate  business  process 
reengineering  assertions  were  not  completed  and  the  associated 
results  and  outcomes  have  yet  to  be  reported. 

Source:  GAO  analysis  of  DOD  data. 


In  addition,  the  Office  of  the  Deputy  Chief  Management  Officer  has  yet  to 
determine  and  follow  a  strategic  approach  to  managing  its  human  capital  needs, 
thus  limiting  its  ability  to,  among  other  things,  effectively  address  the  act’s 
requirements.  Collectively,  these  limitations  put  the  billions  of  dollars  spent 
annually  on  approximately  2,100  business  system  investments  that  support  DOD 
functions  at  risk.  GAO’s  previous  recommendations  to  the  department  have  been 
aimed  at  accomplishing  these  and  other  activities  related  to  the  business 
systems  modernization.  However,  to  date,  the  department  has  not  implemented 
29  of  the  63  recommendations  that  GAO  has  made  in  these  areas. 

According  to  DOD  officials,  recent  turnover,  changes  to  the  act’s  requirements 
significantly  expanding  the  number  of  systems  subject  to  certification,  and  the 
short  time  frame  for  implementing  the  new  investment  review  process  contributed 
to  the  aforementioned  weaknesses.  Until  DOD  implements  GAO 
recommendations  and  addresses  the  weaknesses  described  in  this  report,  it  will 
be  challenged  in  its  ability  to  manage  the  billions  of  dollars  invested  annually  in 
modernizing  its  business  system  investments. 

_ United  States  Government  Accountability  Office 
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GAO 


U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


441  G  St.  N.W. 
Washington,  DC  20548 


May  17,  2013 

Congressional  Committees 

For  decades,  the  Department  of  Defense  (DOD)  has  been  challenged  in 
modernizing  its  business  systems.  In  1995,  we  designated  the 
department’s  business  systems  modernization  program  as  high  risk 
because  of  its  vulnerability  to  fraud,  waste,  abuse,  and  mismanagement, 
and  because  of  missed  opportunities  to  achieve  greater  efficiencies;  and 
we  continue  to  designate  it  as  such  today.1  In  addition,  we  have  reported2 
that  significant  potential  exists  for  identifying  and  avoiding  costs 
associated  with  duplicative  functionality  across  DOD  business  system 
investments,  which  includes  approximately  2,100  systems.3  DOD’s 
business  systems  include  those  for  personnel,  financial  management, 
healthcare,  and  logistics,  and  cost  the  department  billions  of  dollars  each 
year. 

Since  designating  this  area  as  high  risk  in  1995,  we  have  made  a  series 
of  recommendations  aimed  at  strengthening  DOD’s  institutional  approach 


^AO,  High-Risk  Series:  An  Update,  GAO-13-283  (Washington,  D.C.:  February  2013). 

2GAO,  Opportunities  to  Reduce  Potential  Duplication  in  Government  Programs,  Save  Tax 
Dollars,  and  Enhance  Revenue,  GAO-1 1-31 8SP  (Washington,  D.C.:  Mar.  1,  2011); 
Follow-up  on  2011  Report:  Status  of  Actions  Taken  to  Reduce  Duplication,  Overlap,  and 
Fragmentation,  Save  Tax  Dollars,  and  Enhance  Revenue,  GAO-12-453SP  (Washington, 
D.C.:  Feb.  28,  2012);  and  GAO-13-283. 

3Defense  Information  Technology  Portfolio  Repository  dated  March  22,  2013. 
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to  modernization  and  reducing  the  risk  associated  with  key  investments.4 
Further,  Congress  included  provisions  in  the  Ronald  W.  Reagan  National 
Defense  Authorization  Act  for  Fiscal  Year  2005  (NDAA)5  as  amended, 
that  were  consistent  with  our  recommendations.  Specifically,  section  332 
of  the  act,  as  amended,  requires  the  department  to,  among  other  things, 
(1)  develop  a  business  enterprise  architecture  (BEA)  to  cover  all  defense 
business  systems,  (2)  develop  a  transition  plan  for  implementing  the 
architecture,  (3)  identify  systems  information  in  its  annual  budget 
submission,  (4)  establish  a  systems  investment  approval  and 
accountability  structure  along  with  an  investment  review  process,  and  (5) 
certify  and  approve  any  business  system  program  costing  in  excess  of  $1 
million.  The  act  directs  the  Secretary  of  Defense  to  submit  an  annual 
report  to  the  congressional  defense  committees  on  DOD’s  compliance 
with  certain  requirements  of  the  NDAA  not  later  than  March  15  of  each 
year,  through  2016. 

The  act  directed  us  to  report  to  these  congressional  committees — within 
60  days  of  DOD’s  report  submission — an  assessment  of  the  department’s 
actions  to  comply  with  the  requirements  of  the  act.  Accordingly,  our 
objective  was  to  assess  the  actions  by  DOD  to  comply  with  the  act. 


4GAO,  DOD  Business  Systems  Modernization:  Long-standing  Weaknesses  in  Enterprise 
Architecture  Development  Need  to  Be  Addressed,  GAO-05-702  (Washington,  D.C.:  July 
22,  2005);  DOD  Business  Systems  Modernization:  Billions  Being  Invested  without 
Adequate  Oversight,  GAO-05-381  (Washington,  D.C.:  Apr.  29,  2005);  DOD  Business 
Systems  Modernization:  Limited  Progress  in  Development  of  Business  Enterprise 
Architecture  and  Oversight  of  Information  Technology  Investments,  GAO-04-731  R 
(Washington,  D.C.:  May  17,  2004);  DOD  Business  Systems  Modernization:  Important 
Progress  Made  to  Develop  Business  Enterprise  Architecture,  but  Much  Work  Remains, 
GAO-03-1018  (Washington,  D.C.:  Sept.  19,  2003);  Business  Systems  Modernization: 
Summary  of  GAO’s  Assessment  of  the  Department  of  Defense’s  Initial  Business 
Enterprise  Architecture,  GAO-03-877R  (Washington,  D.C.:  July  7,  2003);  Information 
Technology:  Observations  on  Department  of  Defense’s  Draft  Enterprise  Architecture, 
GAO-03-571  R  (Washington,  D.C.:  Mar.  28,  2003);  DOD  Business  Systems  Modernization: 
Improvements  to  Enterprise  Architecture  Development  and  Implementation  Efforts 
Needed,  GAO-03-458  (Washington,  D.C.:  Feb.  28,  2003);  and  Information  Technology: 
Architecture  Needed  to  Guide  Modernization  of  DOD’s  Financial  Operations,  GAO-01 -525 
(Washington,  D.C.:  May  17,  2001). 

5Pub.  L.  No.  108-375,  §  332,  118  Stat.  1811,  1851-1856  (2004),  as  amended  by  the 
NDAA  for  Fiscal  Year  2013,  Pub.  L.  No.  112-239,  126  Stat.  1632  (2012),  the  NDAA  for 
Fiscal  Year  2012,  Pub.  L.  No.  112-81,  125  Stat.  1298  (2011),  and  the  NDAA  for  Fiscal 
Year  2010,  Pub.  L.  No.  111-84,  123  Stat.  2190  (2009)  (codified  in  part  at  10  U.S.C.  § 

2222.  Hereafter,  we  refer  to  the  provisions  of  10  U.S.C.  §  2222,  including  its  amendments, 
as  the  NDAA  or  'the  act.'). 
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To  accomplish  the  objective,  we  reviewed  and  analyzed  the  latest  version 
of  DOD’s  BEA,  enterprise  transition  plan,  and  investment  management 
policies  and  procedures,  using  our  prior  reports  as  a  baseline.6  To  assess 
the  system  information  in  the  budget  submission,  we  analyzed  and 
compared  information  contained  in  the  department’s  system  that  is  used 
to  prepare  its  budget  submission  with  information  in  the  department’s 
authoritative  business  systems  inventory.  To  address  the  investment 
management  provisions  of  the  act,  we  reviewed  the  department’s  report 
to  Congress,  which  was  submitted  on  March  20,  2013,  and  its  associated 
documentation.  We  also  reviewed  the  documentation  used  to  support 
BEA  and  business  process  reengineering  compliance  assertions  for  fiscal 
year  2013  certifications  for  four  case  studies — Air  Force  Integrated 
Personnel  and  Pay  System  (AF-IPPS),  Integrated  Personnel  and  Pay 
System-Navy  (IPPS-N),  Integrated  Personnel  and  Pay  System-Army 
(IPPS-A),  and  the  Integrated  Electronic  Health  Record  system  (iEHR) — 
and  compared  them  to  DOD’s  BEA  and  business  process  reengineering 
guidance  and  our  business  process  reengineering  assessment  guide.  In 
addition,  we  reviewed  our  guidance  and  reports  on  federal  agencies’ 
workforce  planning  and  human  capital  management  efforts  to  determine 
the  extent  to  which  the  Office  of  the  Deputy  Chief  Management  Officer 
had  implemented  a  strategic  approach  to  human  capital  management. 

We  also  analyzed  documentation  relative  to  corrective  actions  taken  and 
planned  on  our  prior  open  recommendations. 

We  conducted  this  performance  audit  at  DOD  and  military  department 
offices  from  October  2012  to  May  2013,  in  accordance  with  generally 
accepted  government  auditing  standards.  Those  standards  require  that 


6GAO,  DOD  Business  Systems  Modernization:  Governance  Mechanisms  for 
Implementing  Management  Controls  Need  to  be  Implemented,  GAO-12-685  (Washington, 
D.C.:  June  1 , 2012);  GAO,  Department  of  Defense:  Further  Actions  Needed  to 
Institutionalize  Key  Business  System  Modernization  Management  Controls,  GAO-1 1-684 
(Washington,  D.C.:  June  29,  201 1 );  Business  Systems  Modernization:  Scope  and  Content 
of  DOD’s  Congressional  Report  and  Executive  Oversight  of  Investments  Need  to  Improve, 
GAO-10-663  (Washington,  D.C.:  May  24,  2010);  DOD  Business  Systems  Modernization: 
Recent  Slowdown  in  Institutionalizing  Key  Management  Controls  Needs  to  Be  Addressed, 
GAO-09-586  (Washington,  D.C.:  May  18,  2009);  DOD  Business  Systems  Modernization: 
Military  Departments  Need  to  Strengthen  Management  of  Enterprise  Architecture 
Programs,  GAO-08-519  (Washington,  D.C.:  May  12,  2008);  Business  Systems 
Modernization:  Department  of  the  Navy  Needs  to  Establish  Management  Structure  and 
Fully  Define  Policies  and  Procedures  for  Institutionally  Managing  Investments,  GAO-08-53 
(Washington,  D.C.:  Oct.  31,  2007);  and  Business  Systems  Modernization:  Air  Force 
Needs  to  Fully  Define  Policies  and  Procedures  for  Institutionally  Managing  Investments, 
GAO-08-52  (Washington,  D.C.:  Oct.  31, 2007). 


Page  3 


GAO-13-557  DOD  Business  Systems  Modernization 


we  plan  and  perform  the  audit  to  obtain  sufficient,  appropriate  evidence  to 
provide  a  reasonable  basis  for  our  findings  and  conclusions  based  on  our 
audit  objectives.  We  believe  that  the  evidence  obtained  provides  a 
reasonable  basis  for  our  findings  and  conclusions  based  on  our  audit 
objective.  Additional  details  on  our  objective,  scope,  and  methodology  are 
contained  in  appendix  1. 

Background 

DOD  is  one  of  the  largest  and  most  complex  organizations  in  the  world.  In 
support  of  its  military  operations,  the  department  performs  an  assortment 
of  interrelated  and  interdependent  business  functions,  such  as  logistics 
management,  procurement,  health  care  management,  and  financial 
management.  Yet,  we  have  previously  reported  that  the  DOD  systems 
environment  that  supports  these  business  functions  is  overly  complex  and 
error  prone,  and  is  characterized  by  (1)  little  standardization  across  the 
department,  (2)  multiple  systems  performing  the  same  tasks,  (3)  the 
same  data  stored  in  multiple  systems,  and  (4)  the  need  for  data  to  be 
entered  manually  into  multiple  systems.7  For  fiscal  year  2013,  the 
department  requested  about  $6.3  billion  for  its  business  system 
investments  and  about  $16.4  billion  for  its  IT  infrastructure  investments.8 
According  to  the  department’s  systems  inventory,9  its  environment  is 
composed  of  approximately  2,100  business  systems,  including  223  for 
acquisition,  9  for  defense  security  enterprise,  8  for  enterprise  IT 
infrastructure,  285  for  financial  management,  675  for  human  resources 
management,  265  for  installations  and  environment,  653  for  logistics  and 
materiel  readiness,  and  10  for  security  cooperation. 

DOD  currently  bears  responsibility,  in  whole  or  in  part,  for  15  of  the  30 
programs  across  the  federal  government  that  we  have  designated  as  high 
risk.10  Seven  of  these  areas  are  specific  to  the  department,11  and  eight 

7GAO,  DOD  Financial  Management:  Implementation  Weaknesses  in  Army  and  Air  Force 
Business  Systems  Could  Jeopardize  DOD’s  Auditability  Goals,  GAO-12-134  (Washington, 
D.C.:  Feb.  28,  2012). 

8The  DOD  IT  infrastructure  segments  account  for  all  business  and  mission  infrastructure. 
Data  from  the  department’s  fiscal  year  2013  budget  request  is  presented  here  because 
DOD  did  not  issue  its  detailed  fiscal  year  2014  budget  request  for  IT  systems  in  time  to  be 
included  in  this  report. 

9Defense  Information  Technology  Portfolio  Repository  dated  March  22,  2013. 

10GAO-1 3-283. 
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other  high-risk  areas  are  shared  with  other  federal  agencies.12 
Collectively,  these  high-risk  areas  relate  to  DOD’s  major  business 
operations  that  are  inextricably  linked  to  the  department’s  ability  to 
perform  its  overall  mission  and  directly  affect  the  readiness  and 
capabilities  of  U.S.  military  forces  and  can  affect  the  success  of  a 
mission.  In  particular,  the  department’s  nonintegrated  and  duplicative 
systems  impair  its  ability  to  combat  fraud,  waste,  and  abuse.13  As  such, 
DOD’s  business  systems  modernization  is  one  of  the  department’s 
specific  high-risk  areas  and  is  essential  for  addressing  many  of  the 
department’s  other  high-risk  areas.  For  example,  modernized  business 
systems  are  integral  to  the  department’s  efforts  to  address  its  financial, 
supply  chain,  and  information  security  management  high-risk  areas. 


Summary  of  NDAA  Congress  included  provisions  in  the  NDAA,  as  amended,  that  are  aimed 

Requirements  at  ensuring  DOD’s  development  of  a  well-defined  BEA  and  associated 

enterprise  transition  plan,  as  well  as  the  establishment  and 
implementation  of  effective  investment  management  structures  and 
processes.14  The  act  requires  DOD  to,  among  other  things, 

•  develop  a  BEA  that  covers  all  defense  business  systems, 

•  develop  an  enterprise  transition  plan  for  implementing  the 
architecture, 

•  identify  systems  information  in  DOD’s  annual  budget  submissions, 


^These  seven  high-risk  areas  include  DOD’s  overall  approach  to  business 
transformation,  business  systems  modernization,  contract  management,  financial 
management,  supply  chain  management,  support  infrastructure  management,  and 
weapon  systems  acquisition. 

12The  eight  governmentwide  high-risk  areas  include  better  managing  climate  change 
risks,  disability  programs,  ensuring  the  effective  protection  of  technologies  critical  to  U.S. 
national  security  interests,  information  systems  and  critical  infrastructure,  information 
sharing  for  homeland  security,  human  capital,  mitigating  gaps  in  weather  satellite  data, 
and  real  property. 

13GAO,  DOD  Business  Systems  Modernization:  Planned  Investment  in  Navy  Program  to 
Create  Cashless  Shipboard  Environment  Needs  to  Be  Justified  and  Better  Managed, 
GAO-08-922  (Washington,  D.C.:  Sept.  8,  2008);  DOD  Travel  Cards:  Control  Weaknesses 
Resulted  in  Millions  of  Dollars  of  Improper  Payments,  GAO-04-576  (Washington,  D.C.: 
June  9,  2004);  Military  Pay:  Army  National  Guard  Personnel  Mobilized  to  Active  Duty 
Experienced  Significant  Pay  Problems,  GAO-04-89  (Washington,  D.C.:  Nov.  13,  2003); 
and  Defense  Inventory:  Opportunities  Exist  to  Improve  Spare  Parts  Support  Aboard 
Deployed  Navy  Ships,  GAO-03-887  (Washington,  D.C.:  Aug.  29,  2003). 

1410  U.S. C.  §2222. 


Page  5 


GAO-13-557  DOD  Business  Systems  Modernization 


•  establish  an  investment  approval  and  accountability  structure  along 
with  an  investment  review  process,  and 

•  not  obligate  appropriated  funds  for  a  defense  business  system 
program  with  a  total  cost  of  more  than  $1  million  unless  the  approval 
authority  certifies  that  the  business  system  program  meets  specified 
conditions.15 

The  act  also  requires  that  the  Secretary  of  Defense  annually  submit  to  the 
congressional  defense  committees  a  report  on  the  department’s 
compliance  with  these  provisions.  DOD  submitted  its  annual  report  to 
Congress  in  March  2013,  describing  steps  taken,  underway,  and  planned 
to  address  the  act’s  requirements. 


DOD’s  Approach  to 
Business  Systems 
Modernization 


The  department’s  approach  to  modernizing  its  business  systems 
environment,  which  is  part  of  DOD’s  overall  effort  to  transform  its 
business  operations,  includes  reengineering  the  business  processes 
supported  by  its  defense  business  systems,  developing  and  using  a  BEA 
and  associated  enterprise  transition  plan,  and  improving  business 
systems  investment  management.  These  efforts  are  to  be  guided  by 
DOD’s  Chief  Management  Officer  and  Deputy  Chief  Management  Officer. 
Specifically,  the  Chief  Management  Officer’s  responsibilities  include 
developing  and  maintaining  a  departmentwide  strategic  plan  for  business 
reform  and  establishing  performance  goals  and  measures  for  improving 
and  evaluating  overall  economy,  efficiency,  and  effectiveness,  and 
monitoring  and  measuring  the  progress  of  the  department.  The  Deputy 
Chief  Management  Officer’s  responsibilities  include  recommending  to  the 
Chief  Management  Officer  methodologies  and  measurement  criteria  to 
better  synchronize,  integrate,  and  coordinate  the  business  operations  to 
ensure  alignment  in  support  of  the  warfighting  mission  and  developing 


15The  act  (10  U.S.C.  §  2222(a))  requires  the  appropriate  precertification  authority  to 
determine  that  a  defense  business  system  modernization  program  (1)  (a)  is  in  compliance 
with  the  enterprise  architecture  and  (b)  has  undertaken  appropriate  business  process 
reengineering  efforts;  (2)  is  necessary  to  achieve  a  critical  national  security  capability  or 
address  a  critical  requirement  in  an  area  such  as  safety  or  security;  or  (3)  is  necessary  to 
prevent  a  significant  adverse  effect  on  a  project  that  is  needed  to  achieve  an  essential 
capability,  taking  into  consideration  the  alternative  solutions  for  preventing  such  an 
adverse  effect.  The  NDAA  for  fiscal  year  2012  requires  that  the  certification  and  approval 
requirements  apply  to  all  business  systems  programs  that  are  expected  to  cost  more  than 
$1  million  over  the  period  of  the  current  Future-Years  Defense  Program,  which  is  the 
department’s  financial  plan  over  a  6-year  period.  Previously,  the  certification  requirement 
only  applied  to  business  system  modernizations  with  a  total  cost  in  excess  of  $1  million. 
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and  maintaining  the  department’s  enterprise  architecture  for  its  business 
mission  area.16 

DOD  has  assigned  roles  and  responsibilities  to  various  governance 
entities  and  positions  related  to  business  systems  modernization.  For 
example,  the  Deputy’s  Management  Action  Group  is  a  senior-level  forum 
that  meets  several  times  a  month  to  discuss  departmentwide 
management  issues,  including  business-related  topics.  This  group  is  to 
convene  as  the  Defense  Business  Systems  Management  Committee 
when  it  reviews  defense  business  system  portfolios.17  Table  1  describes 
selected  roles  and  responsibilities  and  composition  of  key  governance 
entities  and  positions  related  to  business  systems  modernization. 


Table  1:  DOD  Business  Systems  Modernization  Governance  Entities’  Selected  Roles,  Responsibilities,  and  Composition 


Entity 


Roles  and  responsibilities 


Composition 


Deputy’s  Management  Action 
Group/Defense  Business 
Systems  Management 
Committee 


Provide  strategic  direction  and  plans  for  the  business 
mission  area  in  coordination  with  the  warfighting  and 
enterprise  information  environment  mission  areas.3 
Recommend  policies  and  procedures  required  to  integrate 
DOD  business  transformation  and  attain  cross¬ 
department,  end-to-end  interoperability  of  business 
systems  and  processes. 

Review  defense  business  system  portfolios. 

Serve  as  approval  authority  for  business  system 
investments  greater  than  $1  million. 

Establish  policies  and  approve  the  business  mission  area 
strategic  plan,  the  enterprise  transition  plan  for 
implementation  of  business  systems  modernization,  and 
the  BEA. 


Meets  at  the  discretion  of  the  Deputy 
Secretary  of  Defense  and  is  co-chaired 
by  the  Vice  Chairman  of  the  Joint  Chiefs 
of  Staff.  Includes  senior  leadership  in  the 
Office  of  the  Secretary  of  Defense,  as 
appropriate,  such  as  the  Deputy  Chief 
Management  Officer  and  the  DOD  Chief 
Information  Officer.  Also  includes  the 
military  department  chief  management 
officers,  the  heads  of  select  defense 
agencies,  and  participation  by  other 
senior  management,  including  from  the 
Joint  Chiefs  of  Staff  and  the  U.S. 
Transportation  Command. 


16According  to  DOD  officials,  the  business  mission  area  is  responsible  for  ensuring  that 
capabilities,  resources,  and  materiel  are  reliably  delivered  to  the  warfighter.  Specifically, 
the  business  mission  area  addresses  functional  areas  such  as  real  property  and  human 
resources  management. 

17The  Defense  Business  Systems  Management  Committee  was  established  under  10 
U.S.C.  §186,  which  requires  the  department  to  set  up  a  committee  to  review  and  approve 
major  updates  of  the  defense  BEA  and  to  ensure  that  the  obligation  of  funds  for  defense 
systems  modernization  is  consistent  with  the  criteria  set  out  in  10  U.S.C.  §  2222. 
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Entity 

Defense  Business 
Council/Investment  Review 
Board  (IRB) 


Principal  Staff  Assistants 


Precertification  Authority 


Office  of  the  Deputy  Chief 
Management  Officer 


Roles  and  responsibilities 

Oversee  DOD’s  investment  management  process  and 
conduct  portfolio  analysis  in  support  of  the  review  and 
certification  of  covered  defense  business  system 
programs.15 

Review  functional  strategies  developed  by  the  principal 
staff  assistants  and  assess  component  organizational 
execution  plans. 

Recommend  funds  certification  to  the  Deputy’s 
Management  Action  Group/Defense  Business  Systems 
Management  Committee. 

Prioritize  and  approve  changes  for  inclusion  in  the  BEA. 

Support  the  development  and  implementation  of  the 
department’s  end-to-end  framework. 

Senior  advisors  to  the  Secretary  of  Defense  that  assist  in 
policy  development,  planning,  resource  management, 
fiscal,  and  program  evaluation  responsibilities. 

Develop  functional  strategies  that  are  to  describe  business 
functions,  outcomes,  measures,  and  standards  for  their 
respective  business  areas. c 

Ensure  component-level  investment  review  processes 
integrate  with  the  investment  management  system. 

Identify  those  component  systems  that  require  IRB 
certification  and  prepare,  review,  approve,  validate,  and 
transfer  investment  documentation  as  required. 

Assess  and  precertify  business  process  reengineering  and 
architecture  compliance  for  component  systems  submitted 
for  certification  and  annual  review. 

Provide  acquisition  oversight  and  lead  IT  acquisition 
reform. 

Conduct  and  execute  investment  portfolio  reviews. 
Reengineer  and  apply  end-to-end  processes  to  improve 
business  operations  and  support  audit  readiness. 

Build  and  deliver  the  department’s  BEA,  Strategic 
Management  Plan,  and  enterprise  transition  plan. 

Conduct  organizational  assessments  and  develop 
guidance  to  set  forth  the  priority  performance  outcomes  for 
the  department. 

Manage  the  day-to-day  operations  of  the  Office  of  the 
Deputy  Chief  Management  Officer  (e.g.,  human  resources, 
budgeting,  IT). 


Composition 

Chaired  by  the  DOD  Deputy  Chief 
Management  Officer.  Includes,  for 
example,  the  principal  assistant 
secretaries  of  defense  (Acquisition, 
Logistics  and  Materiel  Readiness),  Joint 
Chiefs  of  Staff,  DOD  Chief  Information 
Officer,  Deputy  Comptroller 
(Programs/Budgets),  Chief  Operating 
Officer,  and  military  departments. 


Composed  of  the  Under  Secretaries  of 
Defense  for  defined  functional  areas 
(e.g.,  Comptroller,  Acquisition, 
Technology,  and  Logistics,  Intelligence, 
Policy,  and  Personnel  and  Readiness; 
and  the  DOD  Chief  Information  Officer). 

Chief  Management  Officer  from  the  Air 
Force,  Army,  Navy,  the  director  or  the 
equivalent  from  the  defense  agencies, 
and  those  designated  by  the  DOD 
Deputy  Chief  Management  Officer  as 
appropriate  for  programs  that  support  the 
business  processes  of  more  than  one 
military  department  or  defense  agency. 


Composed  of  five  directorates 
(Investment  and  Acquisition 
Management,  Enterprise  Business 
Integration,  Technology  and  Innovation, 
Planning  and  Performance  Management, 
and  Operations). 


Source:  GAO  analysis  of  DOD  information. 

aDOD  has  five  core  business  missions:  Human  Resources  Management,  Weapon  Systems  Lifecycle 
Management,  Materiel  Supply  and  Service  Management,  Real  Property  and  Installations  Lifecycle 
Management  and  Financial  Management. 

bA  covered  defense  business  system  is  any  defense  business  system  program  that  is  expected  to 
have  a  total  cost  in  excess  of  $1  million  over  the  period  of  the  current  Future-Years  Defense  Program, 
which  is  the  department’s  financial  plan  over  a  6-year  period. 
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Overview  of  DOD’s  Tiered 
Accountability  for  Business 
Systems  Modernization 


Overview  of  DOD’s  BEA 


cDOD  has  eight  functional  areas:  financial  management,  acquisition,  defense  security  enterprise, 
logistics  and  materiel  readiness,  installations  and  environment,  human  resources  management, 
security  cooperation,  and  enterprise  IT  infrastructure. 


Since  2005,  DOD  has  employed  a  “tiered  accountability”  approach  to 
business  systems  modernization.  Under  this  approach,  responsibility  and 
accountability  for  business  architectures  and  systems  investment 
management  are  assigned  to  different  levels  in  the  department.  For 
example,  the  Deputy  Chief  Management  Officer  is  responsible  for 
developing  the  corporate  BEA  and  the  associated  enterprise  transition 
plan.  Each  component  is  responsible  for  defining  a  component-level 
architecture  and  transition  plan  associated  with  its  own  tiers  of 
responsibility  and  for  doing  so  in  a  manner  that  is  aligned  with  (i.e.,  does 
not  violate)  the  corporate  BEA.  Similarly,  program  managers  are 
responsible  for  developing  program-level  architectures  and  plans  and  for 
ensuring  alignment  with  the  architectures  and  transition  plans  above 
them.  This  concept  is  to  allow  for  autonomy  while  also  ensuring  linkages 
and  alignment  from  the  program  level  through  the  component  level  to  the 
corporate  level. 

Consistent  with  the  department’s  tiered  accountability  approach,  the 
NDAA  for  fiscal  year  2008  required  the  secretaries  of  the  military 
departments  to  designate  the  department  under  secretaries  as  Chief 
Management  Officers  with  primary  responsibility  for  business 
operations.18  Moreover,  the  NDAA  for  fiscal  year  2009  required  the 
military  departments  to  establish  business  transformation  offices  to  assist 
their  chief  management  officers  in  the  development  of  their  respective 
comprehensive  business  transformation  plans19  and  to  develop  well- 
defined  enterprisewide  business  systems  architectures  and  transition 
plans. 

DOD’s  BEA  is  intended  to  serve  as  a  blueprint  for  the  department’s 
business  transformation  efforts.  In  particular,  the  architecture  is  to  guide 
and  constrain  implementation  of  interoperable  defense  business  systems 
by,  among  other  things,  documenting  the  department’s  business  functions 
and  activities;  the  information  needed  to  execute  its  functions  and 
activities;  and  the  business  rules,  laws,  regulations,  and  policies 


18Pub.  L.  No.  110-181,  §  904(b),  122  Stat.  3,  274  (Jan.  28,  2008). 
19Pub.  L.  No.  110-417,  §  908,  122  Stat.  4356,  4569  (Oct.  14,  2008). 
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DOD’s  Approach  to  Approving 
Business  System  Investments 


associated  with  its  business  functions  and  activities.  According  to  DOD, 
its  architecture  is  being  developed  using  an  incremental  approach,  where 
each  new  version  of  the  architecture  addresses  business  mission  area 
gaps  or  weaknesses  based  on  priorities  identified  by  the  department.  The 
department’s  BEA  focuses  on  documenting  information  associated  with 
its  15  end-to-end  business  process  areas.  (See  app.  II  for  a  list  and 
description  of  these  end-to-end  processes.) 

The  department  considers  its  current  approach  to  developing  the  BEA 
both  a  “top  down”  and  “bottom-up”  approach.  Specifically,  according  to 
DOD,  the  architecture  focuses  on  developing  content  to  support 
investment  management  and  strategic  decision  making  and  oversight 
(“top  down”)  while  also  responding  to  department  needs  associated  with 
supporting  system  implementation,  system  integration,  and  software 
development  (“bottom  up”). 

Consistent  with  DOD’s  tiered  approach  to  business  systems 
management,  the  department’s  approach  to  developing  its  BEA  involves 
the  development  of  a  federated  enterprise  architecture,  where  member 
architectures  (e.g.,  Air  Force,  Army,  and  Navy)  conform  to  an  overarching 
corporate  or  parent  architecture  and  utilize  a  common  vocabulary.  This 
approach  is  to  provide  governance  across  all  business  systems, 
functions,  and  activities  within  the  department  and  improve  visibility 
across  DOD’s  respective  efforts.  However,  we  recently  reported20  that 
adopting  this  approach  continued  to  be  a  challenge  for  the  department. 
We  noted  that  while  DOD  was  making  improvements,  its  corporate 
architecture  (i.e.,  the  BEA)  had  yet  to  be  federated  through  the 
development  of  aligned  subordinate  architectures  for  each  of  the  military 
departments.  In  this  regard,  we  reported  that  the  military  departments  had 
made  little  or  no  progress.  Moreover,  DOD  had  yet  to  include  common 
definitions  of  key  terms  and  concepts  to  help  ensure  that  these 
architectures  will  be  properly  linked  and  aligned. 

The  NDAA,  as  amended  in  fiscal  year  2012,  included  significant  changes 
to  the  requirements  for  investment  review  and  certification  of  defense 
business  systems.  Specifically,  it  required  DOD  to  establish  a  single 
investment  review  board  chaired  by  the  Deputy  Chief  Management 
Officer,  an  investment  management  process,  and  significantly  expanded 


20GAO-1 3-283. 
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the  scope  of  systems  requiring  certification  to  include  any  business 
system  with  a  total  cost  of  more  than  $1  million  over  the  course  of  the 
department’s  Future-Years  Defense  Program,  regardless  of  type  of 
funding  or  whether  any  development  or  modernization  is  planned. 
Previously,  the  certification  requirement  only  applied  to  business  system 
modernizations  costing  more  than  $1  million  in  modernization  funds  over 
the  life  cycle  of  the  modernization.  According  to  DOD  documentation,  the 
expanded  scope  resulted  in  a  portfolio  review  of  more  than  1,200  defense 
business  systems,  a  nearly  six-fold  increase  in  the  number  of  systems 
reviewed  previously. 

According  to  DOD,  to  address  the  changes,  the  department  implemented 
an  integrated  business  framework  to  manage  its  business  systems  and 
issued  new  guidance  for  reviewing  and  certifying  business  system 
investments  in  June  2012.  Specifically,  the  department  moved  from  a 
system-based  to  a  portfolio-based  approach  for  the  annual  review  and 
certification  of  defense  business  system  investments,  where  investments 
are  grouped  into  eight  functional  areas  and  collectively  certified  as 
portfolios  within  each  of  the  functional  areas.  According  to  DOD,  the 
framework  enables  the  alignment  of  broad  departmental  strategy  and 
functional  business  strategy  with  organizational  investment  decisions  and 
enables  the  review  and  discussion  of  investments  and  outcomes  by  policy 
and  acquisition  decision  makers. 

The  June  2012  guidance  outlines  sequential  process  steps  and  their 
outputs,  with  each  element  informing  next  steps  in  the  process  flow. 

Table  2  shows  the  steps  to  be  taken  in  DOD’s  investment  management 
process. 
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Table  2:  Summary  of  Steps  to  Be  Taken  in  DOD’s  Investment  Management  Process 


Phase 

Summary 

Build  and  review  functional 
strategies 

The  appropriate  business  line  owners  create  functional  strategies  that  define  the  business  outcomes, 
priorities,  measures,  and  standards  for  a  given  functional  area  within  the  DOD  enterprise.  After  the 
strategies  are  reviewed  by  the  department’s  Investment  Review  Board,  the  Defense  Business 

Council,  they  are  released  to  the  components,  becoming  the  foundation  for  the  remaining  phases  in 
the  process. 

Build  and  submit 
organizational  execution  plans 

Each  component  assembles  its  defense  business  system  investments  into  portfolios  organized  by 
functional  area.  Each  component  then  documents  how  their  portfolios  align  with  the  functional 
strategies  in  organizational  execution  plans  that  consist  of  the  following  elements: 

•  Organizational  execution  plan  brief.  The  brief  demonstrates  a  portfolio’s  alignment  to  goals 
and  outcomes  in  the  functional  strategies,  identifies  the  associated  business  value  of  each  goal, 
and  shows  how  goals  will  be  achieved.  It  also  includes  a  road  map  and  corresponding  time  line 
for  achieving  the  desired  end  state  and  budget  data  for  each  investment. 

•  Validated  system  data.  The  components  are  to  ensure  that  the  data  in  the  organizational 
execution  plans  are  accurate  and  complete  to  support  certification  decisions.  As  the  authoritative 
sources  for  DOD  business  IT  investments,  data  to  populate  the  plans  is  pulled  from  the  Defense 
Information  Technology  Portfolio  Repository  (DITPR)3  and  the  Select  and  Native  Programming 
Data  Input  System — Information  Technology  (SNAP-IT). b 

•  Portfolio  certification  request  memorandum.  A  Precertification  Authority  oversees  the 
development  of  each  plan  and,  through  the  issuance  of  a  portfolio  certification  request 
memorandum,  approves  each  plan  for  submission  to  the  Defense  Business  Council. 

Analyze  portfolios 

Completed  organizational  execution  plans  are  analyzed  and  validated  using  four  criteria:  compliance, 
strategic  alignment,  total  cost,  and  utility. 

Approve  and  implement 
portfolios 

The  Defense  Business  Council  makes  certification  recommendations  to  the  Chair,  who  is  responsible 
for  approving  business  systems  certifications.  The  Chair  documents  certification  decisions  in 
investment  decision  memorandums  that  state  whether  an  individual  organizational  execution  plan 
was  certified  (i.e.,  funding  has  been  approved  but  may  be  subject  to  condition(s)  that  restrict  the  use 
of  funds,  a  time  line  for  obligation  of  funds,  or  mandatory  changes  to  the  portfolio  of  business 
systems)  or  not  certified  (i.e.  funding  is  not  approved  due  to  misalignment  with  strategic  direction, 
mission  needs,  or  other  deficiencies).  Expected  outcomes  of  the  certification  process  include  updates 
to  the  department’s  portfolios,  Strategic  Management  Plan,  enterprise  transition  plan,  and  BEA. 

Source:  GAO  analysis  of  DOD  data. 

aDefense  Information  Technology  Portfolio  Repository  (DITPR)  is  the  department’s  authoritative 
business  systems  inventory. 

bSelect  and  Native  Programming  Data  Input  System — Information  Technology  (SNAP-IT)  is  the 
department’s  system  used  to  prepare  its  budget  submission. 


Prior  GAO  Reviews  of 
DOD’s  Business  Systems 
Modernization 


Between  2005  and  2008,  we  reported  that  DOD  had  taken  steps  to 
comply  with  key  requirements  of  the  NDAA  relative  to  architecture 
development,  transition  plan  development,  budgetary  disclosure,  and 
investment  review,  and  to  satisfy  relevant  systems  modernization 
management  guidance.  However,  each  report  also  concluded  that  much 
remained  to  be  accomplished  relative  to  the  act’s  requirements  and 
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relevant  guidance.21  We  made  recommendations  to  address  each  of  the 
areas.  DOD  largely  agreed  with  our  recommendations. 

However,  in  May  2009,  we  reported  that  the  pace  of  DOD’s  efforts  in 
defining  and  implementing  key  institutional  modernization  management 
controls  had  slowed  compared  with  progress  made  in  each  of  the 
previous  4  years,  leaving  much  to  be  accomplished  to  fully  implement  the 
act’s  requirements  and  related  guidance.22  In  addition,  between  2009  and 
2012,  we  found  progress  had  been  made,  but  long-standing  challenges 
we  had  previously  identified  remained  to  be  addressed.23  For  example, 
the  department: 

•  Issued  BEA  versions  that  continued  to  address  the  act’s 
requirements,  but  had  yet  to  federate  (i.e. ,  extend)  the  architecture 
through  the  development  of  aligned  subordinate  architectures  for  each 
of  the  military  departments. 

•  Included  a  range  of  information  for  1 ,657  business  system 
investments  in  its  fiscal  year  2013  budget  submission;  however,  it  did 
not  reflect  about  500  business  systems,  due  in  part  to  the  lack  of  a 
reliable,  comprehensive  inventory  of  all  defense  business  systems. 

•  Had  begun  to  implement  a  business  process  reengineering  review 
process  but  had  not  yet  measured  and  reported  the  results. 

•  Continued  to  describe  certification  actions  in  its  annual  report  for  its 
business  system  investments  as  required  by  the  act,  but  the  basis  for 
these  actions  and  subsequent  approvals  was  supported  with  limited 
information,  such  as  architectural  compliance  assertions  that  were  not 
validated. 

•  Lacked  the  full  complement  of  staff  it  identified  as  needed  to  perform 
business  systems  modernization  responsibilities. 


21  GAO,  DOD  Business  Systems  Modernization:  Progress  in  Establishing  Corporate 
Management  Controls  Needs  to  Be  Replicated  Within  Military  Departments,  GAO-08-705 
(Washington,  D.C.:  May  15,  2008);  DOD  Business  Systems  Modernization:  Progress 
Continues  to  Be  Made  in  Establishing  Corporate  Management  Controls,  but  Further  Steps 
Are  Needed,  GAO-07-733  (Washington,  D.C.:  May  14,  2007);  Business  Systems 
Modernization:  DOD  Continues  to  Improve  Institutional  Approach,  but  Further  Steps 
Needed,  GAO-06-658  (Washington,  D.C.:  May  15,  2006);  and  DOD  Business  Systems 
Modernization:  Important  Progress  Made  in  Establishing  Foundational  Architecture 
Products  and  Investment  Management  Practices,  but  Much  Work  Remains,  GAO-06-219 
(Washington,  D.C.:  Nov.  23,  2005). 

22GAO-09-586. 

23GAO-1 1-684,  GAO-10-663,  and  GAO-09-586. 
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In  2012,  we  concluded  that  DOD’s  progress  in  addressing  the  act’s 
requirements,  its  vision  for  a  federated  architecture,  and  our  related 
recommendations  were  limited,  in  part,  by  continued  uncertainty 
surrounding  the  department’s  governance  mechanisms,  such  as  roles 
and  responsibilities  of  key  organizations  and  senior  leadership  positions. 
Accordingly,  we  made  recommendations  to  address  the  issues  identified. 
DOD  partially  agreed  with  our  recommendations. 

In  addition,  our  most  recent  high-risk  report  noted  that,  while  DOD’s 
capability  and  performance  relative  to  business  systems  modernization 
had  improved,  significant  challenges  remained.24  For  example,  the 
department  had  not  fully  defined  and  established  a  family  of  management 
controls,  such  as  corporate  and  component  business  architectures  and 
business  system  investment  management  processes.  These 
management  controls  are  vital  to  ensuring  that  DOD  can  effectively  and 
efficiently  manage  an  undertaking  with  the  size,  complexity,  and 
significance  of  its  business  systems  modernization,  and  minimize  the 
associated  risks.  Furthermore,  we  also  recently  reported25  that,  in  order  to 
better  identify  and  address  potential  duplication,  DOD  needs  to  develop 
supporting  component  architectures,  align  them  with  its  corporate 
architecture  to  complete  the  federated  BEA,  and  leverage  its  federated 
architecture  to  avoid  investments  that  provide  similar,  but  duplicative, 
functionality  in  support  of  common  DOD  activities. 


Challenges  in 
Addressing  Previously 
Identified  Concerns 
Limit  DOD’s  Ability  to 
Manage  Its  Business 
Systems  Environment 


Over  the  last  year,  DOD  has  taken  a  number  of  steps  to  comply  with  the 
provisions  of  the  NDAA  and  to  satisfy  relevant  system  modernization 
management  guidance.  In  particular,  the  department  released  its 
enterprise  transition  plan  in  December  2012,  followed  by  an  update  to  its 
BEA  (version  10.0)  in  February  2013,  and  its  annual  report  to  Congress  in 
March  2013  describing  important  steps  that  have  been  taken  and  are 
planned  relative  to  the  act’s  requirements.  However,  many  of  the 
challenges  that  we  have  identified  in  prior  years  still  need  to  be 
addressed.  Specifically, 


24GAO-1 3-283. 

25http://www.gao.gov/duplication/action_tracker/1712. 
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•  Even  though  DOD  has  spent  more  than  10  years  and  at  least  $379 
million26  on  developing  its  BEA,  the  department  has  yet  to 
demonstrate  that  it  is  using  it  to  guide  and  constrain  investments  as 
well  as  develop  the  steps  for  federating  its  architecture  throughout  the 
department. 

•  The  department’s  latest  version  of  its  transition  plan  lacked  important 
content,  such  as  time-phased  milestones,  performance  measures, 
and  financial  resource  needs  for  all  business  systems. 

•  DOD  has  yet  to  fully  implement  its  plans  and  establish  a  deadline  for 
reconciling  and  validating  the  completeness  and  reliability  of 
information  used  to  prepare  its  budget  submissions,  as  previously 
recommended. 

•  The  department’s  recently  established  portfolio-based  approach  for 
reviewing  and  certifying  fiscal  year  2013  investments  lacks  an 
established  foundation  and  criteria  and  procedures  for  making 
investment  decisions. 

•  DOD  has  yet  to  ensure  the  accuracy  of  BEA  alignment  through 
validation  of  investments,  complete  appropriate  business  process 
reengineering  assertions  on  all  investments,  and  report  on  the  results 
of  its  efforts. 

•  The  Office  of  the  Deputy  Chief  Management  Officer  has  yet  to  follow  a 
strategic  approach  to  human  capital  management  and  many  of  the 
positions  needed  to  execute  the  department’s  responsibilities  remain 
unfilled. 

•  The  department  has  not  implemented  29  of  the  63  recommendations 
that  we  have  made  since  2005  to  assist  the  department  in  addressing 
the  act’s  provisions  on  the  architecture,  transition  plan,  budgetary 
disclosure,  and  investment  management. 


BEA  Content  Continues  to 
Be  Developed,  but  Use  of 
the  Architecture  Has  Been 
Limited 


The  act  requires  DOD  to  develop  a  BEA  that  covers  all  defense  business 
systems  and  their  related  functions  and  activities.  Among  other  things,  the 
BEA  should  include  data  standards,  policies,  procedures,  and 
performance  measures  that  are  to  be  applied  throughout  the  department 
and  define  a  target  defense  business  systems  computing  environment  for 
the  department’s  major  business  processes.  We  have  also  previously 


26This  figure  only  reflects  data  provided  by  DOD  and  reported  in  GAO-05-702  ($318 
million)  along  with  a  recent  update  on  costs  incurred  by  the  Office  of  the  Deputy  Chief 
Management  Officer  in  developing  the  BEA  between  fiscal  years  2010  and  2013  ($61 
million). The  period  of  time  between  our  June  2005  report  and  the  beginning  of  fiscal  year 
2010  is  not  accounted  for  because  those  costs  were  incurred  by  the  now-disestablished 
Business  Transformation  Agency. 
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reported27  that  the  BEA  is  an  important  tool  for  identifying  potential 
overlap  and  duplication  among  DOD  business  systems. 

DOD  has  continued  to  add  needed  content  to  the  BEA.  Specifically,  the 
department’s  most  recent  version  (10.0),  released  in  February  2013, 
focuses  on  updating  content  within  core  business  areas  such  as  financial 
management  and  human  resources  management  and  making 
improvements  to  the  list  of  laws,  regulations,  and  policies  documented  in 
the  BEA.  This  version  includes  information  aimed  at  meeting  the  act’s 
requirements,  such  as: 

•  Data  standards  intended  to  help  provide  for  a  common  business 
language  for  reporting  and  sharing  financial  information  among  DOD 
business  systems.  For  example,  the  current  version  of  the  BEA 
includes  data  standards  associated  with  DOD’s  Standard  Line  of 
Accounting,  such  as  the  Security  Cooperation  Implementing  Agency 
Code,  which  the  BEA  defines  as  a  single  character  code  that  identifies 
the  military  department  or  agency  that  has  negotiated  or  facilitated  a 
foreign  military  sale  on  behalf  of  the  U.S.  government.  Identifying  such 
standards  is  important  for  improving  interoperability  among  business 
systems  and  enhancing  traceability  between  budgets  and 
expenditures  to  help  achieve  audit  readiness. 

•  Business  rules  to  help  ensure  compliance  with  the  laws,  regulations, 
and  policies  incorporated  in  the  BEA.  For  example,  one  business  rule 
added  to  inform  the  development  of  human  resources  systems  states 
that  the  secretary  concerned  must  not  pay  an  inactive  National  Guard 
member  for  inactive  duty  training.  Such  business  rules  help  to  ensure 
that,  among  other  things,  the  department’s  business  systems 
consistently  implement  DOD  policies  and  guidance. 

•  Performance  measures  that  show  linkages  among  DOD  activities  and 
metrics  from  the  department’s  Strategic  Management  Plan.  For 
example,  the  BEA  shows  that  the  Manage  Civilian  Staff  Acquisition, 
Manage  Labor  Relations,  and  Perform  Workforce  Analysis  business 
activities  are  linked  to  the  Strategic  Management  Plan  metric 
associated  with  improving  the  cycle  time  for  hiring  external  civilians. 
This  is  important  for  meeting  the  act’s  requirements  associated  with 
performance  measures  and  to  enable  traceability  of  BEA  content  to 
the  Strategic  Management  Plan. 


27GAO-1 1-318SP  and  GAO-12-453SP. 


Page  16 


GAO-13-557  DOD  Business  Systems  Modernization 


The  NDAA  also  requires  the  department  to  have  a  BEA  that  federates  to 
all  defense  components  (e.g.,  Air  Force,  Army,  and  Navy).  Toward  this 
end,  since  2005, 28  DOD  has  initiated  a  number  of  efforts  to  federate  the 
BEA,  including  developing  a  federated  strategy,  conducting  federation 
pilots,  and  using  semantic  web  technologies29  to  provide  visibility  across 
its  respective  business  architecture  efforts.  Among  other  things,  DOD’s 
approach  to  federation  called  for  the  corporate  BEA,  each  end-to-end 
business  process  area  (e.g.,  Procure-to-Pay),  and  each  DOD  component 
(e.g.,  Air  Force,  Army,  Navy)  to  establish  a  common  vocabulary  and  for 
the  programs  and  initiatives  associated  with  these  areas  to  use  this 
vocabulary  when  developing  their  respective  system  and  architecture 
products.  Yet,  we  have  previously  reported  that  the  federation  strategy 
lacked  adequate  details  needed  to  achieve  the  department’s  goals  and 
accordingly  recommended  that  that  the  department’s  efforts  to  develop 
the  federated  BEA  be  supported  by  an  architecture  management  plan 
that  incorporated  the  details  needed  to  execute  its  Business  Mission  Area 
Federation  strategy.30 

However,  according  to  the  Chief  Architect,  the  department’s  approach  to 
federation  is  currently  being  reassessed,  adding  that  future  approaches 
for  architecture  federation  might  not  involve  the  use  of  semantic  web 
technologies.  Instead,  officials  stated  that  the  department  is  working  with 
its  Component  Collaboration  Forum,  which  includes  representatives  from 
the  military  departments  and  defense  agencies,  to  generate 
recommendations  for  how  to  proceed  with  federating  the  architecture. 
Officials  from  the  Office  of  the  Deputy  Chief  Management  Officer  added 
that  the  department  has  experienced  recent  turnover  in  key  positions, 
which  has  contributed  to  the  challenges  in  developing  a  federated 
approach  for  its  architecture. 

Although  the  department  has  spent  more  than  10  years  and  at  least  $379 
million  on  developing  its  BEA  and  has  been  attempting  to  extend  the  BEA 
to  its  components  since  2005,  it  has  yet  to  do  so.  Further,  the  architecture 
continues  to  lack  critical  content  that  is  needed  to  achieve  the 


28GAO-06-219. 

pq 

Semantic  web  technologies  include  the  use  of  nonproprietary,  open  standards  and 
protocols  to  develop  DOD  architectures  to  allow  users  to,  among  other  things,  locate  and 
analyze  needed  architecture  information  across  the  department. 

30GAO-07-451 . 


Page  17 


GAO-13-557  DOD  Business  Systems  Modernization 


department’s  vision  for  an  architecture  that  can  be  used  to  guide, 
constrain,  and  enable  interoperable  business  systems. 

For  example: 

•  Business  activities  have  not  been  defined  at  a  level  that  allows  for  the 
identification  of  potential  overlap  and  duplication.  For  example,  while 
the  architecture  includes  business  activities,  such  as  perform 
reporting,31  provide  geospatial  visualization  services,32  and  manage 
environmental  liability  information,33  it  does  not  decompose  these 
activities  to  a  lower  level  (e.g.,  describe  the  activities  involved  in 
managing  geospatial  information  and  services).  As  a  result,  programs 
cannot  identify  whether  they  address  such  lower-level  activities  and, 
thereby,  provide  the  department  with  information  that  it  could  use  to 
identify  duplicative  or  overlapping  capabilities. 

•  While  the  architecture,  including  the  transition  plan,  includes  a  listing 
of  target  business  systems,  it  has  yet  to  identify  target  systems 
applicable  to  each  operational  activity.  For  example,  the  architecture 
does  not  identify  any  systems  that  support  the  perform  legal 
investigation  operational  activity.  In  addition,  while  it  identifies  families 
of  systems  (i.e.,  groups  of  similar  systems),  the  architecture  has  not 
specified  all  of  the  individual  systems  within  each  family  of  systems. 
Specifically,  the  BEA  identifies  the  Personnel  and  Pay  Management 
family  of  systems,  which  is  a  group  of  systems  intended  to  support 
many  existing  personnel  and  pay  processes,  but  information  about 
specific  systems  within  this  group  of  systems  (e.g.,  the  Army  and  Air 
Force’s  Integrated  Personnel  and  Pay  Systems)  have  yet  to  be 
defined.  Moreover,  each  of  the  systems  within  this  family  of  systems 
is  intended  to  replace  multiple  existing  systems,  which  are  not 
identified  in  the  architecture.  As  a  result,  the  architecture  does  not 
include  sufficient  information  to  allow  the  identification  of  systems  that 


31The  perform  reporting  business  activity  includes  receiving  financial  and  management 
reporting  requirements,  preparing  the  information  product,  and  distributing  the  finished 
product  to  the  requestor. 

op 

The  provide  geospatial  visualization  activity  manages  geospatial  IT  visualization 
applications  and  services  for  DOD  business  missions  and  creates  geospatial  information 
products. 

oo 

The  manage  environment  liability  information  business  activity  identifies  and  values 
environmental  liabilities,  and  prepares  environmental  liability  information  for  reporting 
purposes. 
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might  support  similar  business  activities,  potential  duplication  and 
overlap,  or  inform  decision  making  about  reusing  or  integrating 
existing  systems. 

•  While  DOD’s  Strategic  Management  Plan  for  fiscal  years  2012  and 
2013  states  that  DOD  was  to  have  completed  mapping  its  Hire-to- 
Retire  and  Procure-to-Pay  business  processes  by  the  end  of  fiscal 
year  2012,  the  department  has  not  done  so.  Specifically,  while  the 
BEA  provides  a  description  of  these  two  end-to-end  business 
processes,  their  related  business  capabilities  have  yet  to  be  defined. 
For  example,  business  capabilities34  have  not  yet  been  defined  for  the 
perform  manpower  planning,35  manage  recruiting,36  and  manage 
labor  relations37  business  activities,  which  are  part  of  the  Hire-to- 
Retire  end-to-end  business  process.  In  addition,  business  capabilities 
have  not  yet  been  defined  for  the  create  purchase  requisition,38  award 
procurement  instrument,39  and  manage  disbursements40  business 
activities.  Without  this  information,  the  department  risks  not  being  able 
to  optimize  business  processes  and  the  systems  that  support  them  to 
reduce  business  systems  spending. 

According  to  the  department’s  Chief  Architect,  some  of  the  missing  detail 

is  not  intended  to  be  in  the  corporate  architecture  (i.e.,  BEA).  Instead,  the 


34Among  other  things,  DOD  describes  capabilities  as  information  that  describes  the  ability 
to  achieve  a  desired  effect  under  specific  standards  and  conditions  through  a  combination 
of  means  and  ways  to  perform  a  set  of  tasks.  For  example,  the  account  for  personnel 
business  capability  is  associated  with,  among  other  things,  accounting  for  time,  absence, 
and  labor,  and  performing  leave  and  absence  administration. 

35The  perform  manpower  planning  business  subprocess  is  associated  with  projecting 
manpower  requirements,  identifying  the  mission  list,  and  developing  policy  and  procedure 
guidance,  to  support  preparation  of  the  DOD  budget,  and  includes  both  budgetary  and 
executionary  requirements. 

36The  manage  recruiting  business  subprocess  is  associated  with  managing  the 
recruitment  process  for  applicants  who  apply  to  the  Armed  Forces. 

37The  manage  labor  relations  subprocess  is  associated  with  managing  the  relationship 
between  the  agency  and  its  unions  and  bargaining  units. 

oo 

The  create  purchase  requisition  business  subprocess  relates  to  the  initiation  and 
management  of  requests  for  the  purchase  of  goods  and/or  services. 

oq 

The  award  procurement  instrument  business  subprocess  results  from  the  execution  of 
an  approved  acquisition/sourcing  plan  and  results  in  the  execution  of  contractual 
documentation  and  the  legal  obligation  of  funds. 

40The  manage  disbursements  business  subprocess  supports  all  activities  necessary  to 
execute  the  payment  process  for  transactions  that  have  been  authorized  for  payment. 
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content  should  be  documented  in  the  component  architectures  that  will 
align  with  the  corporate  architecture  as  part  of  the  department’s  approach 
to  federation.  However,  given  that  the  department  is  reassessing  its 
approach  to  federation,  it  remains  unclear  as  to  when  and  how  some  of 
this  content  will  be  reflected  as  part  of  its  federated  vision  for  an 
architecture  that  covers  all  defense  business  systems  and  their  related 
functions  and  activities.  Until  the  department  does  so,  it  will  be  limited  in 
its  ability  to  use  the  architecture  as  a  tool  for  managing  its  business 
systems  investments  by,  for  example,  more  effectively  identifying  areas  of 
potential  duplication  and  overlap. 


Enterprise  Transition  Plan 
Content  Has  Been 
Improved,  but  Is  Still 
Missing  Important  Content 


The  act  further  calls  for  the  development  of  an  enterprise  transition  plan 
that  covers  all  defense  business  systems  and  includes  a  listing  of  the 

(a)  new  systems  that  are  expected  to  be  needed  to  complete  the  defense 
BEA,  along  with  each  system’s 


•  time-phased  milestones, 

•  performance  measures, 

•  financial  resource  needs,  and 

•  risks  or  challenges  to  integration  into  the  BEA; 

(b)  legacy  systems  that  will  not  be  part  of  the  defense  BEA,  together  with 
the  schedule  for  terminating  those  legacy  systems,  that  provides  for 
reducing  the  use  of  these  systems  in  phases;  and 


(c)  legacy  systems  that  will  be  a  part  of  the  target  defense  business 
systems  computing  environment,  as  well  as  a  strategy  for  making  the 
modifications  to  those  systems  that  will  be  needed  to  ensure  that  such 
systems  comply  with  the  defense  BEA,  including  time-phased  milestones, 
performance  measures,  and  financial  resource  needs. 


The  department’s  enterprise  transition  plan,  issued  in  December  2012, 
includes  more  than  1,200  covered  defense  business  systems  due  to 
changes  in  the  NDAA  for  fiscal  year  2012.  Along  with  its  related 
documentation,  such  as  the  organizational  execution  plans  and  the  OMB 
exhibit  300s,41  the  enterprise  transition  plan  includes  milestone  data,  such 


41  Each  year,  agencies  submit  the  exhibit  300  to  OMB  to  justify  each  request  for  a  major  IT 
investment. 
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as  termination  schedule  information  on  legacy  systems  that  will  not  be 
part  of  the  defense  BEA. 

While  DOD  has  begun  to  improve  its  enterprise  transition  plan, 
improvements  are  needed  to  fully  address  the  act’s  requirements. 

•  The  plan  includes  acquisition  program  milestone  information  (e.g., 
initial  operational  capability)  in  organizational  execution  plans  and  the 
department’s  functional  strategies.  However,  this  documentation  does 
not  provide  milestones  for  all  business  systems  expected  to  be  part  of 
the  target  architecture,42  nor  does  it  provide  the  capability  to  easily 
view  milestone  information  that  might  be  contained  in  other  sources, 
such  as  the  Defense  Acquisition  Management  Information  Retrieval 
system.43  For  example,  the  organizational  execution  plans  do  not 
identify  the  complete  set  of  individual  systems  and  their  associated 
milestones.  In  addition,  while  selected  exhibit  300s  include  information 
about  key  system  events,  such  as  system  start  and  end  dates,  these 
exhibit  300s  only  capture  those  business  systems  that  are  classified 
as  a  major  investment  and  do  not  reflect  the  complete  set  of  systems 
that  will  be  part  of  the  target  environment. 

•  The  department’s  functional  strategies  include  metrics  associated  with 
improving  each  of  the  functional  areas  (e.g.,  financial  management), 
but  do  not  include  performance  measures  for  each  target  business 
system.  According  to  DOD  officials,  milestones  will  be  addressed  in 
future  versions  of  the  plan  through  the  development  of  a  systems 
evolution  description,  known  as  an  SV-8.44  Officials  from  the  Office  of 
the  Deputy  Chief  Management  Officer  also  stated  that  they  plan  to 
include  acquisition  program  measures  and  other  related  measures  in 
future  versions  of  the  plan. 

•  The  plan  includes  information  about  fiscal  year  2013  funding 
approved  under  the  department’s  business  system  investment  review 


42DOD  considers  these  systems  to  be  “core”  systems.  Officials  from  DOD  components 
designated  every  defense  business  system  as  either  a  core  or  legacy  system,  according 
to  the  department’s  business  system  investment  management  guidance.  This  guidance 
defines  covered  legacy  systems  as  those  that  are  to  be  terminated  within  36  months  of  the 
date  of  their  certification  approval  and  all  other  systems  are  considered  core  systems. 

43The  Defense  Acquisition  Management  Information  Retrieval  system  is  a  DOD  initiative 
that  provides  enterprise  visibility  to  Acquisition  program  information. 

44 According  to  DOD’s  architecture  framework,  an  SV-8  describes  the  life  cycle  of  systems 
and  how  they  change  over  time.  In  addition,  an  SV-8  can  be  combined  with  other 
architecture  information  to  show  how  an  organization’s  capabilities  change  over  time. 
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process  for  each  business  system  expected  to  be  part  of  the  target 
business  architecture.  However,  the  plan  does  not  include  information 
about  financial  resource  needs  for  each  business  system.  DOD 
officials  stated  that  future  versions  of  the  plan  are  to  include  financial 
information  (i.e.,  financial  resource  needs)  obtained  from  querying 
data  currently  located  in  the  department’s  data  repositories. 

•  The  plan  and  related  documentation  do  not  discuss  each  system’s 
risks  or  challenges  to  integration  into  the  BEA.  Instead,  DOD  officials 
referred  to  the  constraints  and  enablers  documented  in  each 
organizational  execution  plan.  For  example,  the  Department  of  the 
Navy’s  financial  management  organizational  execution  plan  cited 
“reaching  enterprise-wide  agreements  on  the  reduction  of  multiple  and 
often  competing  business  processes  that  are  counterproductive  and 
ineffective”  and  the  Defense  Logistics  Agency’s  organizational 
execution  plan  cited  “synchronization  with  other  DOD  efforts”  as  a 
challenge.  In  addition,  the  Army’s  acquisition  organizational  execution 
plan  cited  “end-to-end  business  process  development”  as  an  enabler 
and  the  Air  Force’s  acquisition  organizational  execution  plan  cited 
“use  of  the  Defense  Information  Systems  Agency’s  modern 
capabilities  and  enterprise  services”  as  an  enabler.45  However,  these 
high-level  constraints  and  enablers  do  not  address  risks  or  challenges 
to  integration  associated  with  each  system.  DOD  officials  stated  that 
future  versions  of  the  plan  will  include  information  about  integration 
risks  and  challenges  to  integration  from  investment  decision 
memoranda  and  future  organizational  execution  plans. 

•  The  plan  includes  a  listing  of  legacy  systems  that  will  not  be  part  of 
the  defense  BEA  along  with  the  associated  termination  dates.  For 
example,  the  plan  includes  termination  dates  for  1 10  legacy  systems. 
However,  the  plan  does  not  discuss  how  these  systems  will  be 
terminated  in  phases.  Instead,  it  only  includes  system  end  dates, 
which  do  not  reflect  a  phased  approach  to  termination.  DOD  officials 
stated  that  future  versions  of  the  plan  are  expected  to  include  this 
information  using  department  and  program-level  data  to  identify 
schedules  and  dependencies.  In  addition,  termination  dates  described 
in  the  plan  are  not  always  accurate.  Specifically,  10  legacy  systems 
have  termination  dates  that  occur  prior  to  the  beginning  of  fiscal  year 
2014.  DOD  officials  acknowledged  that  not  all  dates  are  accurate  and 
attributed  the  inaccuracies  to  the  time  required  to  improve  the  data 


45The  organizational  execution  plan  refers  to,  among  other  things,  leveraging  common 
services  for  identity,  security,  communication,  and  workflow. 
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associated  with  the  large  number  of  new  systems  subject  to  the  act’s 
requirements.  They  stated  that  they  are  actively  working  to  improve 
the  data  and  noted  that  the  plan  includes  notes  about  data  quality, 
including  notes  indicating  that  some  life-cycle  end  dates  occur  in  the 
past. 

According  to  officials  in  the  Office  of  the  Deputy  Chief  Management 
Officer,  an  internal  assessment  was  conducted  in  late  2012  to  identify 
gaps  in  the  enterprise  transition  plan,  and  efforts  are  ongoing  to  develop  a 
plan  for  what  is  required  to  address  these  gaps  and  the  act’s 
requirements. 

In  addition  to  these  gaps,  the  department  has  also  yet  to  address  the 
act’s  requirement  for  including  a  listing  of  the  legacy  systems  that  will  be 
a  part  of  the  target  defense  business  systems  computing  environment  as 
well  as  selected  information  about  those  systems.  More  specifically, 
although  DOD’s  approach  to  addressing  this  requirement  involves 
identifying  all  of  the  systems  considered  to  be  target  business  systems, 
the  Office  of  the  Deputy  Chief  Management  Officer  has  only  recently 
(April  2013)  defined  more  detailed  expectations  for  the  target  defense 
systems  computing  environment.46  As  such,  it  remains  to  be  seen  how 
this  definition  will  be  implemented  in  a  future  transition  plan.  In  addition, 
while  DOD  does  not  plan  to  make  modifications  to  legacy  systems  that 
will  not  be  part  of  the  target  BEA,  the  existing  enterprise  transition  plan 
documentation  does  not  identify  a  strategy  for  making  the  modifications  to 
legacy  systems  that  will  be  part  of  the  target  BEA  to  ensure  that  they 
comply  with  the  defense  BEA,  including  time-phased  milestones, 
performance  measures,  and  financial  resource  needs.  Moreover, 
regardless  of  how  DOD  defines  a  legacy  system,  it  is  important  for  the 
department  to  include  information  regarding  phased  terminations  of 
existing  systems  to  help  improve  planning  for  the  department’s  future 
business  systems  environment. 


46The  April  2013  guidance  defines  the  target  business  systems  computing  environment  as 
the  “to  be”  environment  consisting  of  (1)  the  core  covered  defense  business  system 
programs  and  related  resources  which  the  DOD  will  use  to  conduct  its  major  business 
processes  and  (2)  the  supporting  enterprise  IT  infrastructure  and  related  resources,  such 
as  networks,  communications,  enterprise  shared  services,  enterprise  information 
assurance,  in  the  enterprise  information  environment  and  other  mission  areas. 
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Beyond  this,  we  have  previously  reported47  that  the  plan  did  not  include 
information  needed  to  understand  the  sequencing  of  systems.  To  its 
credit,  the  department  has  begun  to  use  a  top-down  approach  for 
sequencing  investments  (e.g.,  using  functional  strategies  to  identify 
strategic  initiatives  and  propose  systems  transition).  However,  the 
planned  investments  have  not  been  sequenced  based  on  a  range  of 
important  factors  cited  in  federal  guidance,  such  as  governmentwide  and 
agency-specific  priorities  (e.g.,  open  and  transparent  government), 
dependencies  among  investments,  expectations  about  investment  costs 
and  benefits,  and  emerging  and  available  technological  opportunities 
(e.g.,  cloud  computing). 

Officials  from  the  Office  of  the  Deputy  Chief  Management  Officer 
attributed  the  gaps  in  enterprise  transition  plan  content  to  the  significant 
increase  in  the  number  of  systems  that  the  act  requires  to  be  included  in 
the  plan.  Specifically,  according  to  the  DOD  officials,  the  number  of 
systems  subject  to  the  act’s  requirements  increased  from  269  systems  in 
the  fiscal  year  2012  transition  plan  to  more  than  1,200  in  the  fiscal  year 
2013  plan.  They  stated  that  gaps  in  the  plan’s  existing  content  will  be 
improved  in  future  versions. 

Until  the  department  defines  by  when  and  how  the  enterprise  transition 
plan  will  include  the  complete  set  of  required  information  on  the  full 
inventory  of  investments  across  the  department  (and  does  so  in  a  manner 
that  reflects  consideration  of  the  range  of  variables  associated  with  a  well- 
defined  transition  plan,  such  as  dependencies  among  investments),  it  will 
not  be  able  to  provide  the  department  with  a  sufficient  basis  for 
sequencing  business  capabilities  and  system  functions  to  be  subsumed 
by  a  modern  system. 


47GAO-09-586. 
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Plans  for  Improving  the 
Reliability  of  Business 
System  Investment 
Information  Are  Not  Fully 
Implemented 


The  NDAA  requires  that  DOD’s  annual  IT  budget  submission  include  key 
information  on  each  business  system  for  which  funding  is  being 
requested,  such  as  the  system’s  precertification  authority  and  designated 
senior  official,  the  appropriation  type  and  amount  of  funds  associated  with 
modernization  and  current  services  (i.e.,  operation  and  maintenance), 
and  the  associated  Defense  Business  Systems  Management  Committee 
approval  decisions.48  The  budget  submission  relies  on  business  system 
investment  information  (e.g.,  funds  requested,  mission  area,  and  system 
description)  that  the  components  entered  into  the  department’s  system 
used  to  prepare  its  budget  submission  (SNAP-IT).  In  accordance  with 
DOD  guidance  and  according  to  officials  in  the  Office  of  the  DOD  Chief 
Information  Officer,  the  business  systems  listed  in  SNAP-IT  should  match 
the  systems  listed  in  the  department’s  authoritative  business  systems 
inventory — Defense  Information  Technology  Portfolio  Repository 
(DITPR).  We  have  previously  recommended  that  the  department  develop 
and  implement  a  plan  and  establish  a  deadline  for  reconciling  and 
validating  the  completeness  and  reliability  of  information  in  its  two 
repositories,  and  include  information  on  the  status  of  these  efforts  in  the 
department’s  annual  report  in  response  to  the  act.49 


DOD  Chief  Information  Officer  officials  have  reiterated  the  department’s 
commitment  to  integrating  the  two  repositories  and  described  steps  taken 
and  future  steps  toward  achieving  this  end.  For  example,  the  officials 
stated  that  a  planned  investment  portal  that  is  intended  to  present  a 
unified  interface  to  users  for  both  IT  budget  and  system  information  is  in 
development.  Officials  also  stated  that  this  new  portal  is  expected  to 
reach  a  limited  operational  capability  in  May  2013  and  when  planned 
capabilities  are  implemented,  issues  associated  with  previously  reported 
differences  between  the  number  of  business  systems  reflected  in  the  two 
repositories  would  have  been  addressed.  In  addition,  to  prepare  for  the 
planned  integration,  DOD  officials  have  described  steps  it  has  begun  to 
take  to  improve  the  quality  of  the  data  in  the  systems.  For  example, 

DOD’s  March  20  annual  report  pursuant  to  the  NDAA  stated  that  the 
Defense  Business  Council  directed  precertification  authorities  to  update 
and  validate  the  authoritative  data  sources,  and  specific  feedback  on  data 
errors  was  provided  to  the  precertification  authorities.  In  addition,  the 


40 

Due  to  ongoing  budget  negotiations,  DOD  did  not  prepare  its  detailed  fiscal  year  2014 
budget  request  in  time  for  inclusion  in  this  report. 

49GAO-09-586  and  GAO-12-685. 
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report  stated  that  a  community  of  interest  work  group  was  updating  the 
data  collection  repositories  and  removing  conflicting  data  sources. 
Moreover,  DOD  has  developed  detailed  plans  for  the  integrated  portal 
that  includes,  among  other  things,  roles  and  responsibilities, 
requirements,  a  data  dictionary,  business  rules,  and  implementation  time 
frames  for  an  incremental  release,  such  as  achieving  full  operational 
capability  for  its  final  increment  by  March  2015.  Nevertheless,  the  officials 
stated  that  efforts  to  implement  these  planned  capabilities  for  the 
integrated  portal  have  been  impacted  by  changes  in  the  management 
structure  for  DITPR  and  by  requirements  to  rehost  the  repository  due  to  a 
base  realignment  and  closure.  As  such,  it  remains  unclear  as  to  when 
DOD  will  be  able  to  fully  implement  its  plans  for  having  an  integrated 
portal. 

Until  DOD  fully  implements  its  plans  and  establishes  a  deadline  for 
completing  the  repository  integration  and  validating  its  completeness  and 
reliability,  as  we  have  previously  recommended,  it  will  not  be  able  to 
ensure  that  annual  budget  submissions  and  oversight  decisions  relying 
on  these  repositories  will  be  based  on  complete  and  accurate  information. 


Steps  to  Establish  a 
Portfolio-Based 
Investment  Management 
Approach  Have  Begun,  but 
Much  Work  Remains 


The  act,  as  amended,  requires  DOD  to  establish  an  investment  approval 
and  accountability  structure  along  with  an  investment  review  process.  In 
addressing  the  act’s  provisions,  DOD  has  taken  steps  to  establish  a 
portfolio-based  approach  to  certifying  defense  business  systems, 
including  the  establishment  of  a  corporate-level  Investment  Review  Board 
(IRB)  to  oversee  the  approach  and  guidance  for  selecting,  controlling,  and 
evaluating  the  investment  portfolio.  However,  it  has  yet  to  fully  establish 
the  foundation  for  its  new  portfolio-level  investment  management  process 
or  the  criteria  and  procedures  for  making  portfolio-based  investment 
decisions. 


Investment  Review  Board  to 
Oversee  Portfolio-Based 
Investment  Management 
Process  Has  Been  Established 


Section  901  of  the  NDAA  for  fiscal  year  2012  requires  the  establishment 
of  a  single  IRB  to  oversee  the  department’s  investment  review  process 
and  conduct  periodic  reviews  of  all  covered  defense  business  systems 
programs.  The  act  also  calls  for  the  IRB  to  be  established  by  the  DOD 
Deputy  Chief  Management  Officer,  with  representation  from  appropriate 
officials,  including  the  DOD  Chief  Information  Officer  and  appropriate 
senior  officials  for  the  functions  and  activities  supported  by  the  defense 
business  systems  under  review.  Consistent  with  the  act,  our  IT 


Page  26 


GAO-13-557  DOD  Business  Systems  Modernization 


Foundation  for  Portfolio-Level 
Investment  Management  Has 
Yet  to  Be  Fully  Established 


Investment  Management  framework50  also  calls  for  the  establishment  of 
an  enterprisewide  investment  board  that  is  composed  of  senior 
executives  from  IT  and  business  units. 

To  comply  with  the  new  requirements  of  the  act,  the  department  changed 
its  investment  management  governance  structure  by  establishing  an 
enterprisewide  IRB  and  eliminating  the  multiple,  functionally  oriented 
boards  used  previously.  Specifically,  in  October  2012,  DOD  established 
the  Defense  Business  Council  as  the  department’s  enterprisewide,  single 
IRB  and  assigned  the  council  overall  responsibility  for  governing  the 
optimization  of  DOD’s  enterprisewide  business  environment  and  the 
improvement  of  the  department’s  business  activities  and  management 
structures.  In  accordance  with  the  NDAA  and  our  investment 
management  framework,  the  council  is  chaired  by  the  DOD  Deputy  Chief 
Management  Officer  and  includes  members  such  as  the  Deputy  DOD 
Chief  Information  Officer,  the  Assistant  Secretary  of  Defense 
(Acquisition),  the  Deputy  Under  Secretary  of  Defense  (Installations  and 
Environment),  the  Office  of  Joint  Chiefs  of  Staff,  and  representatives  from 
the  Air  Force,  Army,  and  the  Department  of  the  Navy.  By  establishing  an 
IRB  composed  of  appropriate  senior  executives,  the  likelihood  of  making 
decisions  that  reflect  the  needs  of  the  enterprise  is  increased. 

According  to  DOD’s  guidance,  the  foundation  of  the  new  investment 
management  process  is  the  set  of  six  functional  strategies  that  prioritize 
and  identify  the  enterprise’s  immediate  needs  while  providing  direction  for 
each  business  area.  The  guidance  requires  that  the  functional  strategies 
demonstrate  alignment  of  goals  with  enterprisewide  business  goals 
identified  in  DOD’s  Strategic  Management  Plan.  Other  critical  elements 
that  must  be  included  are  the  vision  for  each  business  area,  the  goals  that 
will  lead  to  accomplishment  of  the  vision,  and  the  expected  outcomes  for 
each  goal.  To  track  progress  toward  the  accomplishment  of  stated  goals 
and  outcomes,  the  guidance  calls  for  performance  measures  that  include 
baseline  and  target  measures,  and  provide  a  rationale  for  the  identified 
targets.  The  completed  functional  strategies  are  to  inform  the 


50GAO’s  IT  Investment  Management  framework  provides  a  method  for  evaluating  and 
assessing  how  well  an  agency  is  selecting  and  managing  its  IT  resources.  The  framework, 
which  describes  five  progressive  stages  of  maturity  that  an  agency  can  achieve  in  its 
investment  management  capabilities,  was  developed  on  the  basis  of  our  research  into  the 
IT  investment  management  practices  of  leading  private-  and  public-sector  organizations. 
See  GAO,  Information  Technology  Investment  Management:  A  Framework  for  Assessing 
and  Improving  Process  Maturity,  GAO-04-394G  (Washington,  D.C.:  March  2004). 
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components’  technology  portfolios,  provide  the  context  for  portfolio-based 
investment  decisions,  become  the  basis  for  the  department’s  enterprise 
transition  plan,  help  to  expand  and  refine  the  department’s  Strategic 
Management  Plan,  and  become  part  of  the  BEA. 

For  the  fiscal  year  2013  certification  of  defense  business  systems,  six 
functional  strategies  that  covered  seven  of  DOD’s  eight  functional  areas 
were  developed.51  In  accordance  with  the  investment  management 
guidance,  each  of  the  six  included  many  of  the  critical  elements  called  for 
in  the  guidance.  Specifically, 

•  Five  of  the  six  functional  strategies  demonstrated  linkages  to  business 
goals  in  DOD’s  Strategic  Management  Plan,  but  one  did  not.  As  a 
result,  it  lacked  the  direction  needed  to  guide  efforts  for  developing  a 
portfolio  that  adheres  to  the  strategy’s  tenets  while  ensuring  the 
successful  execution  of  the  department’s  mission. 

•  Each  of  the  six  included  the  vision  and  goals  of  the  corresponding 
functional  area,  but  one  functional  strategy  did  not  include  expected 
outcomes  for  all  functional  area  goals.  Without  fully  documented 
outcomes,  the  department  cannot  adequately  assess  progress 
against  goals,  take  corrective  action  when  needed,  and  demonstrate 
tangible  improvements. 

•  Four  of  the  six  did  not  have  all  performance  measures  in  place  for 
assessing  progress  toward  achieving  stated  goals.  In  addition,  none 
of  the  six  included  performances  measures  that  reflected  all  of  the  key 
attributes  identified  in  DOD’s  guidance.  Specifically,  of  the  72 
performance  measures  documented  in  the  functional  strategies,  none 
included  all  of  the  following:  a  baseline  measure,  a  target  against  the 
baseline,  and  a  rationale  for  the  target.  Without  these  key  attributes, 
the  performance  measures  cannot  function  as  a  standard  for 
comparison  and  provide  a  clear  indication  of  DOD’s  progress  towards 
its  goals. 

According  to  DOD  officials,  the  functional  strategies  were  not  expected  to 
be  fully  mature  for  the  fiscal  year  2013  certification  review  cycle;  instead, 
initial  versions  were  to  highlight  areas  in  which  additional  work  was 


51According  to  DOD  officials,  a  functional  strategy  was  not  created  for  the  department’s 
security  cooperation  functional  area  because  it  does  not  currently  have  any  defense 
business  systems.  Additionally,  they  determined  that  it  would  be  more  useful  to  create  a 
single  combined  strategy  for  the  acquisition  functional  area  and  logistics  and  materiel 
readiness. 
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Criteria  and  Procedures  for 
Making  Portfolio-Based 
Investment  Decisions  Have  Not 
Been  Fully  Defined 


needed  and  updates  would  reflect  lessons  learned  as  well  as 
departmental  priorities,  goals,  and  objectives.  For  the  fiscal  year  2014 
certification  cycle,  the  six  functional  strategies  were  revised.  To  the 
department’s  credit,  this  resulted  in  the  following  improvements: 

•  All  of  the  six  demonstrated  linkages  to  the  business  goals  in  DOD’s 
Strategic  Management  Plan. 

•  Five  of  the  six  had  fully  documented  performance  measures  and  all 
six  identified  one  or  more  expected  outcome  for  each  functional  area 
goal. 

Nevertheless,  DOD  has  not  established  performance  measures  in  which 
all  key  attributes  identified  in  DOD  guidance  are  present  and,  thus,  the 
department  may  lack  a  clear  indication  of  progress  toward  achieving 
shared  goals.  Without  fully  developed  functional  strategies  that  include  all 
of  the  critical  elements  identified  in  DOD’s  guidance,  the  fiscal  year  2013 
certification  reviews  lacked  the  foundation  necessary  for  ensuring  that 
sound  investment  decisions  were  aligned  with  enterprisewide  goals  and 
priorities.  Further,  the  department’s  ability  to  present  a  complete 
enterprise  business  strategy  that  can  be  used  to  update  the  strategic 
management  and  enterprise  transition  plans  as  well  as  the  business 
enterprise  architecture  may  be  limited. 

Investment  portfolios  are  integrated,  enterprisewide  collections  of 
investments  that  are  evaluated  and  managed  collectively  based  on 
common  criteria.  Taking  a  portfolio  perspective  enables  an  organization 
to  consider  its  investments  in  a  more  comprehensive  and  integrated 
manner  so  that,  collectively,  the  investments  optimally  address  the 
organization’s  mission,  strategic  goals,  and  objectives.  Managing  IT 
investments  as  portfolios  also  allows  an  organization  to  determine  its 
priorities  and  make  decisions  about  which  projects  to  begin  funding  and 
continue  to  fund  based  on  analyses  of  the  relative  organizational  value 
and  risks  of  all  projects,  including  projects  that  are  proposed,  under 
development,  and  in  operation. 

Our  investment  management  framework  requires  that  an  organization 
have  documented  policies  and  procedures  for  reviewing,  evaluating,  and 
improving  portfolio  performance,  including  the  use  of  a  predetermined 
performance  threshold  for  analyzing  actual  versus  expected  performance 
(e.g.,  more  than  XX  percent  over  expected  cost) — a  major  factor  in 
defining  remedial  actions  for  underperforming  investments.  Our 
framework  also  highlights  the  importance  of  examining  costs  incurred; 
benefits  attained;  current  schedule;  accuracy  of  project  reporting;  and 
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risks  that  have  been  mitigated,  eliminated,  or  accepted  to  date  as  part  of 
assessing  portfolio  performance.  Altogether,  these  assessments  ensure 
that  the  overall  portfolio  provides  the  maximum  benefits  at  a  desired  cost 
and  at  an  acceptable  level  of  risk. 

DOD’s  investment  management  guidance  identifies  four  criteria  and 
specifies  the  associated  assessments  that  are  to  be  conducted  when 
reviewing  and  evaluating  components’  organizational  execution  plans  in 
order  to  make  a  portfolio-based  investment  decision.  Table  3  provides  a 
summary  of  the  criteria  and  associated  assessments  that  DOD  uses  to 
make  a  certification  decision. 


Table  3:  Summary  of  Criteria  and  Associated  Assessments  for  Making  Certification 
Decisions 


Criteria 

Assessment 

Compliance 

The  Precertification  Authority’s  memorandum  stating  that  each  applicable 
defense  business  system  is  compliant  with  the  relevant  requirements  of 
the  NDAA  for  fiscal  year  2012. 

Strategic 

alignment 

The  degree  of  alignment  to  strategic  goals  and  missions  documented  in 
the  department’s  Strategic  Management  Plan  and  the  functional 
strategies. 

The  degree  to  which  investments  are  being  managed  in  accordance  with 
capital  planning  and  investment  control  practices3  and  DOD’s  Better 
Buying  Power  guidance.15 

Utility 

The  portfolio’s  ability  to  deliver  required  capabilities  for  a  given  function, 
to  demonstrate  interoperability  with  other  systems,  and  to  show 
scalability  to  support  additional  users  or  new  features  in  the  future. 

Total  cost 

Cost  in  the  context  of  the  defense  business  system’s  life-cycle  phase  and 
the  estimated  return  on  investment. 

Source:  GAO  analysis  of  DOD  data. 

aCapital  planning  and  investment  control  is  an  IT  portfolio-driven  management  process  for  ongoing 
identification,  selection,  control,  and  evaluation  of  investments.  The  process  attempts  to  link  budget 
activities  and  agency  strategic  priorities  with  achieving  specific  IT  program  modernization  outcomes. 

bDOD's  Better  Buying  Power  guidance,  issued  in  September  2010,  is  intended  to  improve  affordability 
and  cost  control,  competition,  and  management  of  services  acquisitions,  among  other  things.  Among 
the  specific  actions  called  for  are  increasing  competition  in  services  contracting  and  reducing  the  use 
of  high-risk  contract  vehicles. 


However,  the  guidance  does  not  specify  a  process  for  conducting  an 
assessment  or  call  for  the  use  of  actual  versus  expected  performance 
data  and  predetermined  thresholds.  These  thresholds,  according  to  our 
investment  management  framework,  should  be  established  to  evaluate 
portfolio  performance.  Without  a  defined  approach  that  incorporates  these 
practices,  the  department  is  limited  in  its  ability  to  compare  investments  to 
one  another  within  and  across  the  functional  areas  to  help  ensure  that  its 
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collections  of  investments  optimally  address  the  organization’s  mission 
and  goals. 

In  addition,  the  guidance  does  not  call  for  assessments  to  be  conducted 
in  four  key  areas  identified  in  our  framework:  benefits  attained;  current 
schedule;  accuracy  of  project  reporting;  and  risks  that  have  been 
mitigated,  eliminated,  or  accepted  to  date.  The  framework  stresses  that  it 
is  important  to  examine  all  of  these  aspects  since,  among  other  things, 
some  investments  may  exceed  performance  expectations  (e.g.,  delivering 
at  lower  cost,  in  less  time,  and  with  better  benefits  than  expected).  In 
these  cases,  the  board  may  wish  to  accelerate  an  investment’s  funding  or 
schedule,  reallocate  resources  within  the  overall  portfolio,  or  make  some 
other  type  of  adjustment. 

Further,  the  guidance  does  not  call  for  the  organizational  execution  plans 
to  include  critical  information  for  conducting  assessments  associated  with 
the  strategic  alignment,  utility,  and  total  cost  criteria.  More  specifically,  it 
does  not  require  the  plans  to  include  information  on  alignment  with  the 
capital  planning  and  investment  control  practices  and  Better  Buying 
Power  guidance;  interoperability  among  systems  and  system  scalability  to 
support  additional  users  or  new  features  in  the  future;  and  cost  in 
relationship  to  return  on  investment.  Without  this  critical  information,  the 
department  cannot  ensure  that  certification  decisions  are  being  made  in 
accordance  with  established  criteria. 

DOD  officials  stated  that,  although  assessment  decisions  may  appear 
subjective,  the  reviews  were  based  on  analytics  focused  on  the  four 
criteria  and  the  individual  expertise  of  the  business  leaders.  However, 
according  to  our  framework,  board  members  should  apply  their  judgment 
and  knowledge  to  the  evaluation  of  investment  data  using  documented 
procedures  and  established  criteria. 

To  its  credit,  the  department  issued  revised  guidance  in  April  2013  for  the 
fiscal  year  2014  certification  of  defense  business  systems  that  is  intended 
to  mature  the  investment  management  process  and  reflect  lessons 
learned  from  the  fiscal  year  2013  certification  process.  While  it  is  too  soon 
to  evaluate  the  department’s  updated  approach  to  business  system 
investment  management,  we  plan  to  evaluate  DOD’s  progress  in  defining 
and  implementing  its  updated  investment  review  processes  in  our  fiscal 
year  2014  report  on  defense  business  systems  modernization.  As  DOD 
continues  to  mature  its  investment  management  process,  it  is  important 
that  criteria  and  procedures  for  portfolio-level  management  are  fully 
established  and  that  all  information  needed  to  conduct  portfolio 
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evaluations  is  provided  in  key  documents  to  ensure  that  its  investments 
are  selected  and  controlled  in  a  manner  that  best  supports  its  mission 
needs. 


Investment  Portfolios 
Certified,  but  Compliance 
Assertions  Must  be 
Implemented  More 
Effectively 


The  act  also  requires  that,  prior  to  the  obligation  of  funds,  investments 
involving  business  programs  with  a  total  cost  in  excess  of  $1  million  over 
the  period  of  the  Future-Years  Defense  Program,  whether  appropriated  or 
non-appropriated,  be  certified  to  meet  specific  conditions  defined  in  the 
act,  such  as  demonstrating  compliance  with  DOD’s  BEA  and  whether 
appropriate  business  process  reengineering  efforts  have  been 
undertaken. 

DOD’s  March  2013  report  describes  the  department’s  continuing  efforts  to 
improve  the  efficiency  and  effectiveness  of  its  business  operations 
through  the  use  of  its  Integrated  Business  Framework,  actions  it  will  take 
moving  forward  related  to  the  evolution  of  its  BEA,  and  the  results  of  its 
actions  to  certify  supporting  IT  investments.  Specifically,  according  to  the 
report,  the  department  received  about  1 ,245  certification  requests  totaling 
more  than  $7  billion  in  eight  functional  areas  from  21  DOD  components.52 
Of  the  1,245  requests  within  investment  portfolios,  the  Defense  Business 
Council  approved  1,231  requests  totaling  about  $6.8  billion  (see  table  4). 


Table  4:  Certification  Summary  for  Fiscal  Year  2013 

Functional  portfolio 

Count  of  certification 
requests  approved 

Sum  of  certification 
requests  approved 

Acquisition 

98 

$256,704,000 

Defense  Security  Enterprise 

8 

$44,868,000 

Enterprise  IT  Infrastructure 

5 

$41,231,000 

Financial  Management 

200 

$785,179,000 

Human  Resources  Management 

435 

$2,919,274,000 

Installations  and  Environment 

133 

$288,561,000 

Logistics  and  Materiel 

Readiness 

348 

$2,436,253,000 

Security  Cooperation 

4 

$1,564,000 

Total 

1,231 

$6,773,634,000 

Source:  DOD’s  2013  Congressional  Report  on  Defense  Business  Operations. 

52According  to  the  March  Congressional  Report,  certification  results  were  reported  as  of 
December  3,  2012. 
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The  report  notes  that  the  investment  review  process  resulted  in  the 
identification  of  99  systems  that  will  be  terminated  within  the  next  36 
months  and  are  not  expected  to  receive  budget  funding  beyond  the  next  3 
years.  In  addition,  investments  within  portfolios  were  identified  that 
required  further  refinement  or  elimination,  which  resulted  in  the  denial  of 
more  than  $300  million  in  funding  to  those  investments.  According  to  the 
annual  report  supplement,  three  investment  requests  were  withheld  until 
a  later  time;  seven  were  not  certified;  and  one  was  certified  but  for  a 
smaller  amount  than  was  originally  requested.  The  report  also  includes 
the  status  of  certifications,  such  as  “certified  with  tasking  action  item”  or 
“conditionally  certified,”  for  each  business  system  investment.  However, 
while  the  report  documents  the  number  of  certifications,  it  does  not 
include  the  relevant  information  about  known  weaknesses  related  to  the 
investment  review  process  and  fiscal  year  2013  certifications,  such  as  the 
lack  of  BEA  and  business  process  reengineering-related  compliance  and 
the  extent  to  which  additional  actions  were  needed  to  ensure  compliance. 

Further,  although  the  report  states  that  the  Defense  Business  Council 
took  actions  to  significantly  improve  both  the  visibility  and  accuracy  of  its 
authoritative  data,  we  were  unable  to  verify  the  number  of  certifications 
and  the  funding  amount  requested  using  authoritative  data  sources,  such 
as  component  organizational  execution  plans  provided  by  DOD.  For 
example,  while  the  department  reported  1,245  certification  requests 
totaling  about  $7.17  billion,  we  were  only  able  to  identify  1,223  requests 
totaling  about  $7.41  billion  (a  net  difference  of  22  certification  requests 
and  about  $240  million  in  requested  funds).  DOD  officials  noted  that 
certain  data  fields  in  the  original  data  sources  were  incomplete,  incorrect, 
or  duplicative  when  the  certification  request  was  submitted.  These 
officials  added  that  the  difference  occurred  because  of  in-process 
changes  or  updates  to  the  original  requests.  They  also  stated  that 
extensive  manual  cleanup  efforts  have  been  ongoing  since  the  beginning 
of  the  investment  review  process.  Although  these  data  cleanup  efforts  are 
ongoing,  the  officials  expected  future  portfolio  reviews  to  have  improved 
data. 

While  data  cleanup  efforts  have  been  significant  in  the  first  year  of  this 
new  investment  review  process  and  are  an  important  part  of  building  the 
department’s  baseline  portfolios  of  investments,  without  providing  the  full 
and  accurate  range  of  information  the  department  faces  increased  risk 
that  it  will  not  be  able  to  provide  effective  oversight  over  its  business 
systems  investments.  Further,  by  not  reporting  the  overall  weaknesses 
associated  with  the  certification  activities,  the  department  does  not 
provide  the  full  range  of  information  that  is  needed  to  permit  meaningful 
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BEA  Compliance  Asserted  for 
Most  of  the  Systems,  but 
Assertions  Continue  to  Lack 
Adequate  Validation 


and  informed  congressional  oversight  of  the  department’s  business 
systems  modernization  efforts. 

Under  the  act,  funds  may  not  be  obligated  for  covered  business  systems 
unless  the  certification  requirements  with  regard  to  BEA  compliance  are 
met.  For  its  fiscal  year  2013  certification  review,  investments  were  to 
follow  BEA  compliance  guidance  issued  in  March  201 1 .  This  guidance 
describes  an  incremental  approach  to  BEA  compliance  requirements, 
including  key  elements  needed  for  asserting  architecture  compliance,53 
such  as  specific  artifacts  and  definitions  for  compliance  or  planned 
compliance.54  Further,  we  have  previously  recommended  that  DOD, 
among  other  things,  explicitly  assign  responsibility  for  validating  program 
BEA  compliance  assessments.55 

DOD  has  certified  most,  but  not  all,  investments  on  the  basis  of 
architecture  compliance.  According  to  DOD,  precertification  authorities 
asserted  BEA  compliance  in  their  respective  organizational  execution 
plans  for  about  89  percent  of  systems  submitted  as  part  of  the  portfolio 
certification  process.  For  example,  for  the  four  investments  that  we 
reviewed,  56  the  military  components  and  the  Tricare  Management  Activity 
assessed  BEA  compliance  for  each  investment  in  accordance  with  DOD 
guidance.  In  addition,  the  appropriate  precertification  authorities  asserted 
(via  their  respective  organizational  execution  plans)  whether  each  system 
was  compliant  with  DOD’s  BEA.  For  example,  the  precertification 
authority’s  request  in  the  Army’s  organizational  execution  plan  asserted 
BEA  compliance  for  the  Integrated  Personnel  and  Pay  System-Army 
(IPPS-A).  As  supporting  documentation  for  this  assertion,  the  program 


53For  fiscal  year  201 3  certification,  investments  could  assert  compliance  to  BEA  8.0  or 
9.0. 

54Generally,  if  a  defense  business  system  has  not  yet  deployed  any  capability  into  an 
operational  environment,  an  investment  is  asserted  as  being  “planned  compliant.” 

55GAO,  DOD  Business  Systems  Modernization:  Key  Navy  Programs'  Compliance  with 
DOD's  Federated  Business  Enterprise  Architecture  Needs  to  Be  Adequately 
Demonstrated,  GAO-08-972  (Washington,  D.C.:  Aug  7,  2008). 

56See  app.  I  for  details  on  our  methodology  related  to  four  case  studies,  which  looked  at 
documentation  used  to  support  BEA  and  business  process  reengineering  compliance 
assertions  for  fiscal  year  2013  certifications  related  to  Air  Force-Integrated  Personnel  and 
Pay  System  (AF-IPPS),  Integrated  Personnel  and  Pay  System-Navy  (IPPS-N),  Integrated 
Personnel  and  Pay  System-Army  (IPPS-A),  and  the  Integrated  Electronic  Health  Record 
system  (iEHR). 
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provided  a  compliance  dashboard  to  document  BEA  alignment, 
consistent  with  DOD  guidance,  and  architecture  artifacts  demonstrating 
alignment  to  specific  architectural  content,  such  as  operational  activities 
described  in  the  BEA. 

However,  the  remaining  1 1  percent  of  systems  were  identified  as  not 
being  in  compliance  with  the  architecture.  DOD  officials  stated  that  no 
system  was  denied  certification  due  to  the  lack  of  BEA  compliance; 
instead,  the  Deputy  Chief  Management  Officer  included  an  action  item  in 
all  investment  decision  memoranda  tasking  components  to  document 
compliance  or  a  plan  to  achieve  BEA  compliance  for  any  business 
systems  identified  by  the  precertification  authority  as  not  being  in 
compliance.  However,  while  the  department  has  asserted  compliance  for 
many  business  systems,  it  has  yet  to  validate  these  assertions,  as  we 
have  previously  recommended. 

Office  of  the  Deputy  Chief  Management  Officer  officials  said  that  the  large 
increase  in  the  number  of  systems  to  be  reviewed  made  it  challenging  to 
complete  a  review  for  all  the  systems.  In  addition,  the  short  time  frame  for 
implementing  the  new  investment  review  process  resulted  in  the  need  to 
include  the  action  items.  They  also  stated  that  they  are  tracking 
responses  to  these  actions  for  each  of  the  portfolios  and  expect  the 
assertions  for  fiscal  year  2014  reviews  to  be  more  complete  than  the 
fiscal  year  2013  reviews. 

With  regard  to  validation,  DOD  officials  stated  that  some  information 
associated  with  the  compliance  process  had  been  validated,  such  as 
information  associated  with  complying  with  DOD’s  Standard  Financial 
Information  Structure.57  However,  the  DOD  Inspector  General  recently 
reported  that  an  assessment  of  an  enterprise  resource  planning  system 
determined,  among  other  things,  that  the  system  inaccurately  asserted 
compliance  based  on  a  future  capability  rather  than  on  the  system’s 
actual  capability  at  the  time  of  the  assessment.58  It  further  stated  that  a 
more  robust  certification  process  was  needed  instead  of  having  the  Office 


57The  Standard  Financial  Information  Structure  is  intended  to  provide  a  standard  financial 
management  data  structure  and  uniformity  throughout  DOD  in  reporting  on  the  results  of 
operations. 

58DOD  Inspector  General,  Enterprise  Business  System  Was  Not  Configured  to  Implement 
the  U.  S.  Government  Standard  General  Ledger  at  the  Transaction  Level,  DODIG  2013- 
057  (Alexandria,  VA:  March  20,  201 3). 
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of  the  Deputy  Chief  Management  Officer  rely  on  a  self-certification 
process  that  did  not  include  validation. 


Officials  from  the  Office  of  the  Deputy  Chief  Management  Officer  have 
stated  their  intent  to  perform  BEA  compliance  validation  for  selected 
defense  business  systems  as  part  of  the  investment  review  process. 
However,  they  have  not  yet  executed  these  activities  and  were  not  able  to 
provide  milestones  for  when  this  would  be  accomplished.  Among  other 
things,  BEA  compliance  is  important  for  helping  to  ensure  that  DOD 
programs  have  been  optimized  to  support  operations.  As  we  and  others 
have  reported,  without  proper  oversight  and  validation  of  compliance 
assertions  and  plans  to  be  compliant,  DOD  programs  are  at  increased 
risk  of  being  defined  and  implemented  in  a  way  that  does  not  sufficiently 
ensure  interoperability  and  avoid  duplication  and  overlap,  which  are  both 
goals  of  the  BEA  and  related  investment  management  approach. 


Appropriate  Business  Process 
Reengineering  Assertions  Were 
Not  Completed  for  the  Majority 
of  Investments  for  Fiscal  Year 
2013  Certifications 


The  act  requires  that  precertification  authorities  determine  that 
appropriate  business  process  reengineering  efforts  be  undertaken  for  the 
investment  prior  to  certification  approval  for  each  covered  business 
system.  To  address  this  requirement,  DOD  issued  guidance  in  April  201 1 
to  implement  its  approach  for  asserting  business  process  reengineering 
compliance,  which  was  intended  to  ensure  that  the  underlying  business 
process  supported  by  the  business  system  modernization  was  as 
streamlined  and  efficient  as  practicable.  This  guidance  required 
submission  of  a  form  that  integrates  questions  in  areas  related  to,  for 
example,  business  process  definition  and  target  process  improvements. 
We  have  also  issued  business  process  reengineering  guidance  that 
recognizes,  among  other  things,  the  importance  of  developing  a  clear 
problem  statement  and  business  case  for  reengineering  and  analyzing 
the  current  and  target  environments.59 


The  four  investments  we  reviewed  generally  followed  the  department’s 
business  process  reengineering  guidance  for  asserting  compliance. 
Specifically,  all  four  programs  we  reviewed — IPPS-A,  IPPS-N,  AF-IPPS, 
and  iEHR —  submitted  business  process  reengineering  forms,  consistent 
with  DOD  guidance,  and  were  asserted  by  their  respective  precertification 
authorities  as  being  in  compliance.  These  programs,  all  of  which  are  large 


59GAO,  Business  Process  Reengineering  Assessment  Guide  (Version  3), 
GAO/AIMD-IO.1.15  (Washington,  D.C.:  May  1997). 
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and  complex  investments,  also  had  documentation,  which  included,  to 
varying  degrees,  a  general  description  of  problems  or  issues  to  be  solved, 
process  maps  that  graphically  documented  the  target  business 
processes,  and  process  maps  that  graphically  documented  the  current 
business  processes  that  were  reengineered. 

However,  limited  documentation  of  root  cause  analyses  was  provided  for 
the  four  investments.  For  example,  iEHR  documented  root  cause 
analyses  for  some,  but  not  for  all  five  capabilities  that  were  the  focus  of  its 
reengineering  efforts.  It  also  did  not  document  the  likelihood  of  each 
probable  cause.  As  we  have  previously  reported,  60  there  is  a  long  history 
of  challenges  in  trying  to  achieve  electronic  health  records  sharing 
between  DOD  and  the  Department  of  Veterans  Affairs,  and  this  analysis 
would  be  a  key  document  for  supporting  the  business  process 
reengineering  assertion.  Without  a  root  cause  analysis  to  trace  symptoms 
back  to  the  underlying  factors,  an  agency  may  determine  inappropriate 
corrective  actions,  including  IT  solutions,  to  address  the  true  problems  of 
business  process  inefficiencies. 

While  precertification  authorities  asserted  business  process  reengineering 
compliance  for  IPPS-A,  IPPS-N,  AF-IPPS,  and  iEHR,  according  to  DOD, 
precertification  authorities  asserted  that  appropriate  business  process 
reengineering  had  been  undertaken  on  only  about  41  percent  of  the 
approximately  1 ,200  systems  for  the  fiscal  year  2013  certification  reviews. 
Department  officials  confirmed  that  no  system  was  denied  certification 
due  to  lack  of  business  process  reengineering  assertion  even  though, 
according  to  the  act,  compliance  is  a  requirement  for  obligation  of  funds. 
As  with  the  BEA  compliance,  DOD  included  an  action  item  in  nearly  all 
investment  decision  memoranda  tasking  components  to  document  a  plan 
to  achieve  business  process  reengineering  compliance  for  any  business 
systems  identified  by  the  precertification  authority  as  not  being  in 
compliance.  According  to  department  officials,  one  of  the  reasons  for  a 
number  of  systems  that  were  not  asserted  for  appropriate  business 
process  reengineering  was  confusion  about  the  extent  to  which  business 
process  reengineering  was  required  for  systems  in  sustainment.61  Also 


60GAO,  Electronic  Health  Records:  Long  History  of  Management  Challenges  Raises 
Concerns  about  VA’s  and  DOD’s  New  Approach  to  Sharing  Health  Information , 
GAO-13-413T  (Washington,  D.C.:  Feb.  27,  2013). 

61Systems  in  sustainment  are  operational,  not  currently  modernizing,  and  are  not  slated 
for  retirement. 
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similar  to  the  BEA  compliance,  officials  from  the  Office  of  the  Deputy 
Chief  Management  Officer  stated  that  the  increase  in  the  number  of 
certification  requests  made  it  challenging  to  complete  a  review  for  all  the 
systems.  In  addition,  they  added  that  the  short  time  frame  for 
implementing  the  new  investment  review  process  resulted  in  the  need  to 
include  the  action  items.  They  also  stated  that  they  are  tracking 
responses  to  these  actions  for  each  of  the  portfolios  and  expect  the 
assertions  for  fiscal  year  2014  reviews  to  be  more  complete  than  the 
fiscal  year  2013  reviews. 

The  department  has  also  begun  to  validate  business  process 
reengineering  assertions.  Specifically,  DOD  updated  its  BPR  guidance  in 
late  September  2012  to  include  actions  to  be  taken  to  review  supporting 
documentation  on  selected  investments.  Beginning  in  late  November 
2012,  the  Office  of  the  Deputy  Chief  Management  Officer  selected  35 
defense  business  systems  from  across  the  department  to  inspect  and 
validate  business  process  reengineering  assertions.  According  to  officials, 
validation  efforts  have  been  labor  intensive  and  will  inform  future  business 
process  reengineering  assertions.  However,  results  are  not  expected  to 
be  reported  to  the  Defense  Business  Council  in  full  until  June  2013  and  it 
is  unclear  what  actions  will  be  taken  as  a  result.  DOD’s  September  2012 
business  process  reengineering  guidance  notes  that  the  Defense 
Business  Council  actions  may  include  levying  conditions  on  defense 
business  systems  or  decertification  of  2013  funds  until  the  business 
system  becomes  compliant,  but  no  actions  had  been  taken  as  of  yet. 

DOD’s  efforts  to  track  action  items  and  validate  precertification 
authorities’  business  process  reengineering  assertions  would  enable  the 
department  to  make  business  system  investment  decisions  based  on 
information  that  is  accurate  and  reliable.  However,  the  department  needs 
to  ensure  that  the  outcomes  of  its  efforts  inform  the  business  process 
reengineering  to  be  performed  on  all  investments  for  the  fiscal  year  2014 
reviews.  Further,  it  should  ensure  that  appropriate  business  process 
reengineering  assertions  have  been  completed  on  all  the  business 
systems  submitted  for  the  fiscal  year  2014  certification  reviews  prior  to 
the  certification  of  funds.  Until  it  does,  the  department  risks  not  being  able 
to  determine  whether  business  processes  are  streamlined  and  efficient  as 
practicable  and  not  complying  with  the  statutory  restriction  on  obligation 
of  funds. 
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DOD  guidance  states  that  business  process  reengineering  should  consist 
of,  among  other  things,  potential  outcomes  related  to: 

•  eliminating  or  reducing  unique  interfaces  to  the  greatest  extent 
possible  and  designing  necessary  information  exchanges  logically 
and  efficiently  and 

•  identifying  appropriate  outcome-based  business  performance 
measures  that  are  consistent  and  linked  to  intended  benefits  of  the 
investment. 

Among  other  things,  our  guidance62  also  emphasizes  the  importance  of 
having  meaningful  performance  measures  to  assess  whether  business 
process  reengineering  activities  actually  achieve  the  intended  results.  For 
example,  business  process  reengineering  results  should  be  focused  on 
providing  better  support  to  an  organization’s  mission,  reducing  costs,  and 
improving  service.  We  have  also  previously  recommended63  that  DOD 
include  in  its  annual  report  to  Congress  the  results  of  the  department’s 
business  process  reengineering  efforts,  such  as  the  department’s 
determination  of  the  number  of  systems  that  have  undergone  material 
process  changes  and  the  number  of  interfaces  eliminated  as  part  of  these 
efforts  (i.e.,  by  program,  by  name). 

For  the  four  investments  we  reviewed,  DOD  reported  that  each  one 
intends  to  reduce  or  eliminate  interfaces.  However,  while  two  of  the 
investments — AF-IPPS  and  IPPS-A — provided  specifics  about  the 
number  of  interfaces  needed  or  the  number  of  interfaces  to  be  reduced, 
none  provided  detailed  assessments  of  existing  interfaces  for  reuse 
opportunities  and  documentation  related  to  the  complexity  of  interfaces. 
For  example,  iEHR  stated  that  it  had  not  yet  performed  an  analysis  on  the 
elimination  of  interfaces  for  any  of  the  business  capabilities  identified  in  its 
supporting  documentation.  In  the  absence  of  such  information,  the 
department  lacks  the  detailed  justification  needed  to  help  it  determine 
whether  the  investment  is  reducing  the  cost  of  fielding  business 
capability. 

Moreover,  the  business  process  reengineering  documentation  concerning 
performance  measures  varied  across  the  four  investments.  For  example, 


Business  Process 
Reengineering  Efforts  Need  to 
Be  Directly  Aligned  with 
Business  Outcomes  and 
Reported 


62GAO/AIMD-10.1.15. 
63GAO-1 2-685. 
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IPPS-N  provided  an  extensive  list  of  metrics  associated  with  its 
reengineering  processes  and  included  baseline  performance  levels  and 
targeted  outcomes,  such  as  specific  decreased  costs  and/or  decreased 
time.  AF-IPPS  included  a  number  of  metrics,  some  of  which  included 
baseline  performance  levels  and  targeted  outcomes.  On  the  other  hand, 
while  IPPS-A  only  specified  targeted  outcomes  related  to  its  architecture 
products,  it  did  not  include  specific  baseline  performance  levels  of  the 
current  process  and  targeted  outcomes  of  the  reengineered  business 
process.  Finally,  iEHR  included  16  out  of  19  performance  metrics  with 
quantitative  outcomes  for  the  five  capabilities’  target  processes;  however, 
performance  baseline  values  were  not  provided  for  any  of  the  current 
processes.  Establishing  a  performance  baseline  from  which  to  measure 
improvements  and  defining  indicators  related  to  quality,  cost,  and  time  is 
key  to  knowing  if  the  reengineered  process  has  produced  the  desired 
effect. 

Further,  while  DOD  included  five  examples  of  business  process 
improvements  in  its  most  recent  annual  report,  it  did  not  indicate  how 
these  examples  were  tied  to  its  business  process  reengineering  efforts 
either  for  a  specific  business  process  or  across  the  larger  end-to-end 
business  processes.  DOD  officials  noted  that  they  intend  to  track  this 
information  better  in  the  future,  but  did  not  describe  how  and  where  this 
information  would  be  reported.  Until  the  department  develops  and  reports 
on  specific  performance  measures,  as  we  have  previously  recommended, 
the  department  and  its  stakeholders  will  not  know  the  extent  to  which 
business  process  reengineering  is  effectively  streamlining  and  improving 
its  business  processes  needed  to  transform  the  department  as  intended. 


Office  of  the  Deputy  Chief 
Management  Officer  Needs 
a  Strategic  Approach  to 
Managing  its  Human 
Capital  Needs 


The  success  of  any  program  depends  on  effectively  leveraging  people, 
processes,  and  tools  to  achieve  defined  outcomes  and  results.  To 
effectively  leverage  people,  they  must  be  treated  as  strategic  assets.  As 
we  have  previously  reported,64  a  strategic  approach  to  human  capital 
management  enables  an  organization  to  be  aware  of  and  prepared  for  its 
current  and  future  human  capital  needs,  such  as  workforce  size, 
knowledge,  skills,  and  training.  To  be  effective,  our  research  shows  that 
such  a  strategic  approach  includes  using  data-driven,  fact-based  methods 


64GAO,  National  Archives  and  Records  Administration:  Oversight  and  Management 
Improvements  Initiated ,  but  More  Action  Needed,  GAO-1 1-1 5  (Washington,  D.C.:  Oct.  5, 
2010). 
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to  (1)  assess  the  knowledge  and  skills  needed  to  execute  a  program;  (2) 
inventory  existing  staff  knowledge  and  skills;  (3)  forecast  the  knowledge 
and  skills  needed  over  time;  (4)  analyze  the  gaps  in  capabilities  between 
the  existing  staff  and  future  workforce  needs,  including  consideration  of 
evolving  program  and  succession  needs  caused  by  turnover  and 
retirement;  and  (5)  formulate  strategies  for  filling  expected  gaps. 

The  Office  of  the  Deputy  Chief  Management  Officer  has  yet  to  determine 
and  follow  such  a  strategic  approach  to  managing  its  human  capital 
needs.  In  particular,  in  addressing  its  staffing  needs,  the  office  has  not 
used  a  documented,  fact-based,  data-driven  methodology  to  assess 
needs  and  existing  capabilities,  nor  has  it  performed  a  gap  analysis  of  the 
number  of  staff  required  and  the  specific  skills  and  abilities  needed  to 
effectively  achieve  its  mission  of  leading  and  enabling  end-to-end 
integration  and  improvement  of  business  operations  in  support  of  national 
security. 

In  2012  we  reported65  that  the  office  had  defined  an  organizational 
structure  and  identified  the  staff  resources  it  would  need  to  fulfill 
responsibilities  associated  with  business  systems  modernization, 
including  leading  efforts  to  develop  the  BEA  and  enterprise  transition 
plan.  This  structure  had  become  effective  in  October  201 1  when  the 
Business  Transformation  Agency  was  formally  eliminated.  However,  it 
had  not  filled  many  of  the  positions  needed  to  execute  these 
responsibilities.  In  particular,  as  of  April  2012,  it  had  filled  only  82  of  its 
planned  139  positions,  with  57  positions  (41  percent)  remaining  unfilled. 
Officials  from  the  Office  of  the  Deputy  Chief  Management  Officer 
attributed  the  unfilled  positions  to,  among  other  things,  challenges 
associated  with  the  length  of  time  between  when  DOD  announced  that 
the  Business  Transformation  Agency,  which  had  previously  addressed 
many  of  the  Office  of  the  Deputy  Chief  Management  Officer’s  current 
functions,  would  be  disestablished  (August  2010)  and  when  the  agency 
was  formally  disestablished  (October  2011).  For  example,  some  staff 
chose  to  seek  employment  elsewhere  due  to  uncertainties  associated 
with  the  transition.  In  response,  we  recommended  that  DOD  include  an 
update  on  the  Office  of  the  Deputy  Chief  Management  Officer’s  progress 
toward  filling  staff  positions  and  the  impact  of  any  unfilled  positions  on  the 


65GAO-1 2-685. 
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ability  of  the  office  to  conduct  its  work  in  its  annual  report  to  Congress  on 
its  business  systems  modernization. 


Since  April  2012,  DOD  has  eliminated  42  full-time  equivalent  positions, 
leaving  97  planned  positions  in  the  Office  of  the  Deputy  Chief 
Management  Officer.  According  to  the  Assistant  Deputy  Chief 
Management  Officer,  40  of  the  42  positions  were  eliminated  as  part  of  the 
Deputy  Secretary  of  Defense’s  efforts  to  find  efficiencies  within  the 
department.  As  of  April  2013,  the  office  reported  having  filled  one  more 
than  the  82  positions  it  had  filled  in  April  2012,  still  leaving  14  positions 
unfilled.  According  to  officials  from  the  Office  of  the  Deputy  Chief 
Management  Officer,  the  department  has  imposed  a  90  percent  hiring 
limit  due  to  sequestration.  As  a  result,  these  officials  stated  that  the  office 
is  not  able  to  fill  1 0  of  the  planned  positions. 

In  March  2013,  DOD  officials  reported  changes  in  the  responsibilities  of 
the  Deputy  Chief  Management  Officer-related  functions.  These  changes 
included  the  realignment  of  responsibilities  among  Deputy  Chief 
Management  Officer  directorates  (e.g.,  moving  the  responsibility  for 
conducting  business  process  reengineering  assessments  from  its 
Investment  and  Acquisition  Management  Directorate  to  its  Enterprise 
Business  Integration  Directorate),  the  elimination  of  several 
responsibilities  (e.g.,  the  office  is  no  longer  executing  enterprise  risk 
assessments  for  Milestone  Decision  Authorities),  and  the  addition  of  other 
responsibilities  (e.g.,  the  Technology  and  Innovation  Directorate  is  now 
responsible  for  developing  and  designing  business  engineering 
methodologies  to  advance  Business  Mission  Area  objectives).  While  the 
office  has  subsumed  many  of  the  responsibilities  that  were  once 
managed  by  DOD’s  Business  Transformation  Agency,  the  Office  of  the 
Deputy  Chief  Management  Officer  has  fewer  staff  and  contractor 
resources.  For  example,  according  to  DOD  budget  documentation,  the 
Business  Transformation  Agency  had  more  than  200  civilian  staff  and 
more  than  600  contract  staff  during  fiscal  year  2010.  However,  as  of  April 
2013,  the  Office  of  the  Deputy  Chief  Management  Officer  consisted  of  five 
directorates  and  was  staffed  with  83  government  employees  and  150 
contractor  staff.  Table  5  shows  the  organizational  components  of  the 
Office  of  the  Deputy  Chief  Management  Officer  and  key  responsibilities. 


Table  5:  Organizational  Components  and  Key  Responsibilities  of  the  Office  of  the 
Deputy  Chief  Management  Officer 


Organizational  component  Key  responsibilities 
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Organizational  component 

Investment  and  Acquisition 
Management  Directorate 


Enterprise  Business 
Integration  Directorate 


Technology  and  Innovation 
Directorate 


Planning  and  Performance 
Management  Directorate 


Operations  Directorate 


Key  responsibilities 

Report  to  Congress  on  progress  and  improvements 
made  in  the  DOD  Business  Mission  Area. 

Develop  the  enterprise  transition  plan. 

Provide  acquisition  oversight  and  lead  IT  acquisition 
reform. 

Conduct  and  execute  investment  portfolio  reviews. 

Reengineer  and  apply  end-to-end  processes  to  improve 
business  operations  and  support  audit  readiness. 

Manage  and  oversee  the  appropriate  end-to-end 
governance  model(s)  and  forum(s). 

Identify  and  analyze  requirements  for  BEA  development. 

Build  and  deliver  the  BEA. 

Develop  and  promote  enterprise  standards  for  business 
architecture. 

Manage  and  execute  BEA  statutory  requirements. 
Develop  the  Strategic  Management  Plan. 

Improve  performance  management  infrastructure  and 
leverage  continuous  process  improvement 
methodologies. 

Conduct  organizational  assessments  and  develop 
organizational  guidance  to  set  forth  the  priority 
performance  outcomes  for  the  department. 

Manage  the  day-to-day  operations  of  the  Office  of  the 
Deputy  Chief  Management  Officer  (e.g.,  human 
resources,  budgeting,  IT). 


Source:  GAO  analysis  based  on  DOD  documentation. 


The  Office  of  the  Deputy  Chief  Management  Officer  has  conducted 
analyses  of  the  functions  that  the  office  is  to  perform,  but  has  not 
conducted  human  capital  analyses.  More  specifically,  to  its  credit,  the 
Office  of  the  Deputy  Chief  Management  Officer  identified  the  functions 
that  each  of  the  office’s  directorates  was  to  perform  as  the  office  was 
being  established.  The  office  also  recently  revisited  these  functions. 
However,  the  Assistant  Deputy  Chief  Management  Officer  stated  that  the 
office  has  not  conducted  human  capital  analyses  and  that  no  plans  exist 
to  analyze  and  address  skill  gaps.  Instead,  the  Assistant  Deputy  Chief 
Management  Officer  stated  that  the  office  focuses  on  addressing  its 
priorities  by  working  with  its  existing  staff  and  leveraging  staff  from 
entities  and  individuals  across  the  department  who  participate  in  related 
entities  (e.g.,  the  Defense  Business  Council)  and  efforts  (e.g.,  end-to-end 
business  process  activities). 

Officials  from  the  Office  of  the  Deputy  Chief  Management  Officer  stated 
that  the  current  level  of  staffing  and  capabilities  is  sufficient  to  effectively 
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achieve  its  mission  and  related  functions.  Nevertheless,  the  department 
has  had  a  number  of  challenges  in  addressing  its  mission  of  achieving  its 
vision  relative  to  developing  a  federated  BEA,  an  enterprise  transition 
plan,  an  investment  management  process,  and  a  strategic  plan,  as 
described  in  this  report  and  our  recent  report  on  business  transformation 
efforts.66  For  example,  we  found  that  while  DOD  had  made  some 
improvements  to  its  Strategic  Management  Plan,  further  refining  the  plan 
to  include  key  information  such  as  performance  measures  that  fully  reflect 
core  activities  needed  to  assess  progress  could  help  DOD  better  prioritize 
and  target  its  reform  efforts  to  address  the  underlying  causes  of  its 
systemic  business  challenges  and  to  achieve  results.  Unless  the  Office  of 
the  Deputy  Chief  Management  Officer  adopts  a  more  strategic  and 
proactive  approach  to  managing  its  human  capital,  it  will  continue  to  be 
challenged  in  its  ability  to  help  ensure  that  the  required  capabilities  and 
mission  value  for  approximately  1,200  business  system  investments — 
totaling  about  $6.8  billion  in  approved  funding  for  fiscal  year  2013 — will  be 
delivered  in  a  timely  and  cost-effective  manner. 


Key  Recommendations 
Have  Not  Yet  Been 
Implemented 


Between  2005  and  2008,  we  made  48  recommendations  to  assist  DOD  in 
developing  a  well-defined  and  useful  BEA  and  business  system 
investment  management  approach  and  in  gaining  better  insight  into  and 
control  over  its  ongoing  business  system  investments,  which  are 
consistent  with  the  NDAA  requirements.  Of  these,  we  reported  that  DOD 
had  implemented  31,  but  17  recommendations  were  not  implemented. 

For  example,  the  department  had  not 

•  Issued  an  architecture  management  plan  that  addresses,  among 
other  things,  how  business  architecture  federation  will  be  governed, 
how  the  alignment  of  component  business  architectures  with 
incremental  versions  of  the  BEA  will  be  achieved,  and  what 
milestones  will  be  used  to  measure  progress  and  results. 

•  Directed  the  responsible  authorities  in  the  department  to  use  the 
program-specific  data  in  the  compliance  assessment  tool  for 
identifying  and  analyzing  potential  overlap  and  duplication,  and  thus 
opportunities  for  reuse  and  consolidation  among  programs  and 
provide  programs  access  rights  to  use  this  functionality. 


66GAO,  Defense  Business  Transformation:  Improvements  Made  but  Additional  Steps 
Needed  to  Strengthen  Strategic  Planning  and  Assess  Progress,  GAO- 13-267 
(Washington,  D.C.:  Feb.  12,  2013). 


Page  44 


GAO-13-557  DOD  Business  Systems  Modernization 


•  Explicitly  assigned  responsibility  for  validating  program  BEA 
compliance  assertions  to  military  departments  and  defense  agencies 
or  issued  guidance  that  describes  the  nature,  scope,  and 
methodology  for  doing  so. 

Since  2009,  we  have  made  15  additional  recommendations  to  assist  DOD 
in  its  business  systems  modernization  efforts.  (See  app.  Ill  for  details  on 
the  status  of  these  1 5  recommendations.)  As  of  April  201 3,  the 
department  had  fully  implemented  3  recommendations,  with  12  remaining 
to  be  addressed.  Specifically,  the  department  has  fully  implemented  our 
recommendations67  to: 

•  include  provisions  in  its  guidance  requiring  certification  submissions  to 
include  any  of  our  open  recommendations  for  program  weaknesses 
as  well  as  the  status  of  actions  to  address  these  recommendations; 

•  disestablish  the  Business  Transformation  Agency  and  complete  the 
transfer  of  its  various  responsibilities  to  other  DOD  entities;  and 

•  ensure  more  accurate  categorizations  of  its  IT  investments. 

In  addition,  the  department  has  begun  to  take  steps  to  address  some  of 
the  remaining  12  recommendations,  including  developing  new  policies 
and  guidance  and  establishing  new  governance  entities,  but  additional 
actions  are  needed  before  the  remaining  recommendations  are  fully 
addressed.  For  example,  in  response  to  our  recommendation68  that  the 
department  establish  a  policy  that  clarifies  the  roles,  responsibilities,  and 
relationships  among  the  various  entities  associated  with  the  development 
of  a  federated  BEA,  the  department  stated  that  it  has  a  number  of  policy 
and  guidance  documents  currently  under  development  to  help  clarify  the 
roles  and  responsibilities  associated  with  development  of  the  BEA  and 
enterprise  architecture  in  general  across  DOD.  For  example,  DOD  has 
established  a  forum  for  discussing  the  development  of  the  federated  BEA 
and  is  currently  working  to  establish  a  BEA  Configuration  Control  Board 
charter,  which  is  intended  to  document  the  roles  and  responsibilities  for 
the  review  and  approval  of  proposed  BEA  content.  However,  DOD  has 
yet  to  issue  these  policy  and  guidance  documents. 

Officials  from  the  Office  of  the  Deputy  Chief  Management  Officer  cited 
various  reasons  for  not  fully  implementing  our  open  recommendations, 


67GAO-1 0-663,  GAO-1 1-684,  and  GAO-12-241. 
68GAO-1 2-685. 
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including  changes  to  the  act’s  requirements  that  significantly  expanded 
the  number  of  systems  subject  to  certification  and  approved  as  compliant 
with  the  BEA  and  having  completed  sufficient  business  process 
reengineering  and  prioritizing  changes  to  balance  competing  priorities.  It 
is  important  that  the  department  move  swiftly  in  doing  so  because  these 
recommendations  are  aimed  at  strengthening  architecture  (and  transition 
planning)  management  activities  and  controlling  ongoing  and  planned 
business  system  investments.  Until  it  does,  the  likelihood  of  sustained 
incremental  improvement  to  its  modernization  management  controls  will 
be  diminished  and  the  means  of  holding  the  department  accountable  for 
such  improvement  will  be  missing. 


Conclusions 


Over  the  last  year,  the  department  has  continued  to  take  a  number  of 
steps  to  comply  with  the  provisions  of  the  act,  including  significant 
changes  to  the  requirements  for  investment  review  and  certification  of 
defense  business  systems.  The  department  has  begun  to  improve  the 
content  of  its  BEA  and  enterprise  transition  plan,  but  additional  content  is 
still  needed  to  fully  address  the  act’s  requirements  and  to  make  the  BEA 
and  plan  more  useful  for  modernizing  the  defense  business  systems 
enterprise.  In  addition,  while  DOD  has  begun  to  improve  the  information  it 
uses  to  help  manage  its  portfolio  of  business  system  investments,  it  has 
yet  to  fully  implement  its  plans  for  validating  the  completeness  and 
reliability  of  the  information.  DOD  has  also  taken  steps,  in  accordance 
with  the  act,  to  establish  a  portfolio-based  approach  to  reviewing  and 
certifying  its  defense  business  systems.  Nevertheless,  the  lack  of  an 
established  foundation  and  criteria  and  procedures  for  making  portfolio- 
based  investment  decisions  will  continue  to  inhibit  the  department’s  ability 
to  make  sound,  portfolio-based  investment  decisions  that  align  with 
mission  priorities.  Further,  DOD  has  yet  to  ensure  the  accuracy  of  BEA 
alignment  through  validation  of  individual  investments.  Moreover,  while 
the  department  has  begun  taking  steps  to  reengineer  its  business 
systems  and  processes,  the  lack  of  appropriate  business  process 
reengineering  assertions  and  the  reporting  of  associated  results  and 
outcomes  puts  the  department  at  risk  of  not  being  able  to  determine 
whether  business  processes  are  streamlined  and  efficient  as  practicable. 
Further,  it  puts  the  department  at  risk  of  noncompliance  with  the  statutory 
restriction  on  obligation  of  funds.  Finally,  the  efforts  of  the  Office  of  the 
Deputy  Chief  Management  Officer  have  been  impacted  by  the  lack  of  a 
strategic  approach  to  managing  its  human  capital  needs.  To  date,  DOD 
has  not  implemented  29  of  the  63  recommendations  in  our  reports  and 
the  modernization  program  remains  on  our  High  Risk  List. 
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To  successfully  accomplish  its  modernization  program,  it  is  important  for 
DOD  to  complete  all  intended  steps  for  implementation  of  the  program: 
improve  its  federated  BEA,  enterprise  transition  plan,  and  its  investment 
management  approach;  validate  the  completeness  and  reliability  of  its 
portfolio  of  business  system  investments;  implement  and  use  the  BEA 
and  business  process  reengineering  compliance  assessments  more 
effectively;  and  address  its  human  capital  needs.  Until  the  department 
takes  such  needed  steps,  it  is  unlikely  that  its  approximately  1,200 
business  system  investments — totaling  about  $6.8  billion  in  approved 
funding  for  fiscal  year  2013 — will  be  managed  in  an  effective  manner  that 
maximizes  mission  performance  while  minimizing  or  eliminating  system 
overlap  and  duplication. 


PpprmrmpnrlatinnG:  fnr  We  are  recommer|ding  that  the  Secretary  of  Defense  direct  the  Deputy 
rtecummeilUcillUIlS  lOI  Chief  Management  Officer  to  take  the  following  six  actions  to  effectively 

Executive  Action  implement  key  components  of  DOD’s  business  systems  modernization 

program: 

•  Define  by  when  and  how  the  department  plans  to  develop  an 
architecture  that  would  extend  to  all  defense  components  and  include, 
among  other  things,  (a)  information  about  the  specific  business 
systems  that  support  BEA  business  activities  and  related  system 
functions;  (b)  business  capabilities  for  the  Hire-to-Retire  and  Procure- 
to-Pay  business  processes;  and  (c)  sufficient  information  about 
business  activities  to  allow  for  more  effective  identification  of  potential 
overlap  and  duplication. 

•  Define  by  when  and  how  the  enterprise  transition  plan  will  include, 
among  other  things,  (a)  milestones,  performance  measures,  and 
funding  plans  for  all  business  systems  expected  to  be  part  of  the 
target  architecture  and  each  system’s  risks  or  challenges  to 
integration;  (b)  time-phased  end  dates  associated  with  terminating 
legacy  systems  in  phases;  (c)  a  listing  of  all  other  defense  business 
systems  (including  systems  that  are  considered  to  be  core  systems) 
that  will  be  a  part  of  the  target  defense  business  systems  computing 
environment  and  a  strategy  for  making  modifications  to  those  systems 
that  will  be  needed  to  ensure  that  they  comply  with  the  defense  BEA, 
including  time-phased  milestones,  performance  measures,  and 
financial  resource  needs;  and  (d)  information  about  how  systems  are 
to  be  sequenced  according  to,  among  other  things,  dependencies 
among  investments. 

•  Ensure  that  the  functional  strategies  include  all  of  the  critical  elements 
identified  in  DOD  investment  management  guidance,  including 
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performance  measures  to  determine  progress  toward  achieving  the 
goals  that  incorporate  all  of  the  attributes  called  for  in  the 
department’s  guidance. 

•  Select  and  control  its  mix  of  investments  in  a  manner  that  best 
supports  mission  needs  by  (a)  documenting  a  process  for  evaluating 
portfolio  performance  that  includes  the  use  of  actual  versus  expected 
performance  data  and  predetermined  thresholds;  (b)  ensuring  that 
portfolio  assessments  are  conducted  in  key  areas  identified  in  our  IT 
investment  management  framework:  benefits  attained;  current 
schedule;  accuracy  of  project  reporting;  and  risks  that  have  been 
mitigated,  eliminated,  or  accepted  to  date;  and  (c)  ensuring  that  the 
documents  provided  to  the  Defense  Business  Council  as  part  of  the 
investment  management  process  include  critical  information  for 
conducting  all  assessments. 

•  Implement  and  use  the  BEA  and  business  process  reengineering 
compliance  assessments  more  effectively  to  support  organizational 
transformation  efforts  by  (a)  disclosing  relevant  information  about 
known  weaknesses,  such  as  BEA  and  business  process 
reengineering  compliance  weaknesses  for  systems  that  were  not 
certified  or  certified  with  qualifications  in  annual  reports  to  Congress; 
(b)  establishing  milestones  by  which  selected  validations  of  BEA 
compliance  assertions  are  to  be  completed;  and  (c)  ensuring  that 
appropriate  business  process  reengineering  assertions  have  been 
completed  on  all  investments  submitted  for  the  fiscal  year  2014 
certification  reviews  prior  to  the  certification  of  funds. 

•  Develop  a  skills  inventory,  needs  assessment,  gap  analysis,  and  plan 
to  address  identified  gaps  as  part  of  a  strategic  approach  to  human 
capital  planning  for  the  Office  of  the  Deputy  Chief  Management 
Officer. 

In  addition,  we  are  recommending  that  the  Secretary  of  Defense  direct 
the  appropriate  authority  to  ensure  that  complete  documentation,  such  as 
root  cause  analyses,  assessments  of  existing  interfaces  for  reuse 
opportunities,  and  performance  metrics  related  to  the  reengineering 
efforts,  is  provided  as  part  of  the  fiscal  year  2014  certification  and 
approval  process  for  the  IPPS-A,  IPPS-N,  AF-IPPS,  and  iEHR 
investments. 

We  further  recommend  that  the  Secretary  direct  the  appropriate  authority 
to  determine  whether  funds  were  properly  obligated  under  10  U.S.C.  § 
2222(a)-(b)  for  systems  for  which  appropriate  business  process 
reengineering  assertions  were  not  completed. 
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Agency  Comments 
and  Our  Evaluation 


In  written  comments  on  a  draft  of  this  report,  signed  by  the  Deputy  Chief 
Management  Officer,  DOD  concurred  with  two  of  our  eight 
recommendations,  partially  concurred  with  three  recommendations,  and 
did  not  concur  with  three  recommendations.  The  department  stated  that  it 
disagreed  with  many  of  the  assertions  put  forward  by  us  and  believed  that 
there  was  a  lack  of  balance  expressed  in  the  report.  It  also  stated  that  the 
report  demonstrated  a  lack  of  understanding  of  the  department’s  new 
investment  management  process  and  the  significant  improvements  it  has 
produced,  such  as  revamping  the  business  mission  area  governance  and 
establishing  an  Integrated  Business  Framework  in  response  to  the  NDAA 
for  Fiscal  Year  2012.  The  department  added  that  these  changes  have 
resulted  in  significant  progress  rather  than  the  slow  improvements 
portrayed  in  the  report.  Further,  the  department  stated  that  while  its 
business  systems  modernization  efforts  will  continue  to  evolve,  it  is 
important  for  the  department  and  Congress  not  to  lose  sight  of  the  bigger 
picture  of  what  they  endeavor  to  achieve  by  focusing  only  on  the  set  of 
challenges  cited  in  our  report.  It  remarked  that  DOD’s  business  mission 
area  is  a  complex,  interconnected  environment  that  is  unprecedented  in 
size.  Finally,  it  stated  that  there  is  significant  focus  in  the  report  on  the 
department’s  failure  to  implement  recommendations  that  are  almost  a 
decade  old  and  that  have  been  overtaken  by  more  recent  events.  DOD’s 
comments  are  reprinted  in  their  entirety  in  appendix  IV. 

In  contrast  to  DOD’s  stated  position,  we  believe  the  report  provides  a 
balanced  view  of  DOD’s  efforts  and  reflects  an  appropriate  level  of 
understanding  of  the  department’s  new,  but  still  evolving,  investment 
management  process.  For  example,  in  our  report,  we  recognized  that 
DOD  had  continued  to  add  needed  content,  such  as  data  standards, 
business  rules,  and  performance  measures  to  its  architecture,  as  called 
for  by  the  act.  However,  we  also  noted  that  while  DOD  had  initiated  a 
number  of  efforts  to  federate  its  BEA  to  the  components,  it  remained 
unclear  as  to  when  and  how  it  would  develop  an  architecture  that  covers 
all  defense  business  systems  and  their  related  functions  and  activities.  As 
another  example,  we  reported  that  to  its  credit,  the  department  had  made 
a  number  of  improvements  to  its  functional  strategies  for  the  fiscal  year 
2014  certification  cycle,  but  also  noted  that  it  had  yet  to  establish 
performance  measures  in  which  all  key  attributes  identified  in  DOD 
guidance  are  present  in  the  strategies. 

Our  report  also  appropriately  focused  on  the  scope  of  work  called  for  by 
the  act’s  reporting  requirements  and  recognized  that  while  DOD  has 
initiated  numerous  management  activities,  such  as  the  Integrated 
Business  Framework,  aimed  at  modernizing  its  business  systems 
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environment,  it  has  also  been  limited  in  its  ability  to  demonstrate  results. 
We  also  recognize  that  DOD’s  business  mission  area  is  a  complex, 
interconnected  environment  that  is  unprecedented  in  size.  In  this  regard, 
our  study  focused  on  the  management  controls,  as  defined  in  the  NDAA 
and  our  prior  recommendations,  which  are  vital  to  ensuring  that  the 
department  can  effectively  and  efficiently  manage  an  undertaking  with  the 
size,  complexity,  and  significance  of  its  business  systems  modernization 
and  minimize  the  associated  risks.  Further,  our  report  appropriately 
focused  on  the  same  challenges  that  DOD  has  been  facing  in 
modernizing  its  business  systems  and  that  prompted  the 
recommendations  in  the  first  place,  regardless  of  their  age.  For  example, 
since  2001 ,  we  have  provided  a  series  of  recommendations  relative  to 
developing  and  using  a  business  enterprise  architecture  and  establishing 
effective  investment  management  controls  to  guide  and  constrain  DOD’s 
multibillion-dollar  business  systems  and  services.  In  addition,  since  2002, 
Congress  has  included  provisions  consistent  with  GAO’s 
recommendations  in  National  Defense  Authorization  Acts,  which  DOD 
has  yet  to  fully  address.  Our  report  also  appropriately  addressed  both  the 
department’s  progress  (to  include  the  implementation  of  34  out  of  63 
recommendations),  as  well  as  the  long-standing  challenges  that  we  had 
previously  identified  and  that  remained  to  be  addressed  during  our  study. 
DOD  has  not  provided  evidence  to  demonstrate  progress  in  addressing  a 
number  of  these  challenges  and  our  related  prior  recommendations  and, 
as  a  result,  the  modernization  program  remains  on  our  High  Risk  List. 
Further,  while  we  support  any  departmental  efforts,  whether  completed  or 
ongoing,  that  would  address  the  challenges  cited  in  our  report,  we  note 
that  DOD  did  not  disagree  with  the  facts  as  presented  in  the  report. 
Therefore,  we  stand  by  our  findings  and  recommendations. 

DOD’s  specific  comments  on  each  recommendation,  along  with  our 
responses  to  its  comments,  follow. 

•  The  department  partially  concurred  with  our  first  recommendation  to 
define  by  when  and  how  it  plans  to  develop  an  architecture  that  would 
extend  to  all  defense  components  and  include  missing  content.  In  this 
regard,  it  agreed  that  plans  for  continuing  to  mature  the  federated 
BEA  should  be  completed.  It  also  noted  that  the  department  had 
already  taken  the  necessary  initial  steps  to  extend  the  BEA  to  all 
defense  components  that  are  institutionalized  via  the  Integrated 
Business  Framework,  which  was  implemented  during  calendar  year 
2012.  In  particular,  the  department  stated  that  its  Integrated  Business 
Framework  links  the  development  of  functional  business  strategies, 
the  identification  of  associated  BEA  changes,  and  the  submission  of 
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organizational  execution  plans  from  the  components.  It  added  that  the 
information  contained  in  the  component-developed  organizational 
execution  plans  constitutes  key  elements  of  the  components’  portions 
of  the  federated  BEA. 

We  recognize  that  the  functional  strategies  and  organizational 
execution  plans  include  information  that  might  be  incorporated  into  the 
BEA,  such  as  information  about  enterprise  standards  and 
performance  measures.  We  also  acknowledged,  in  the  report,  the 
steps  that  DOD  is  taking  to  mature  its  BEA  content,  including 
establishing  a  forum  for  generating  recommendations  regarding  how 
to  proceed  with  the  architecture.  Nevertheless,  while  the  department 
stated  that  it  has  taken  initial  steps  to  federate  the  BEA  to  all  defense 
components,  it  has  yet  to  articulate,  as  recommended,  by  when  and 
how  it  will  fully  achieve  its  federated  BEA.  Thus,  while  the  department 
has  various  efforts  in  place  that  may  contribute  content  to  its 
federated  business  architecture,  as  described  in  the  report,  these 
efforts  have  yet  to  result  in  a  federated  business  architecture  that 
extends  to  all  defense  components,  and  the  architecture  continues  to 
lack  critical  content  needed  to  achieve  the  department’s  vision  for  an 
architecture  that  can  be  used  to  guide,  constrain,  and  enable 
interoperable  business  systems.  Thus,  we  continue  to  stress  full 
implementation  of  this  recommendation. 

•  DOD  partially  concurred  with  our  second  recommendation  to  define 
by  when  and  how  the  enterprise  transition  plan  will  include,  among 
other  things,  milestones,  performance  measures,  and  funding  plans 
for  all  business  systems  expected  to  be  part  of  the  target  architecture 
and  each  system’s  risks  or  challenges  to  integration,  as  identified  in 
our  report.  In  its  comments,  DOD  referred  to  specific  actions  and 
processes  that  it  states  meet  the  act’s  requirements.  For  example, 
DOD  stated  that  milestones,  performance  measures,  and  funding 
plans  for  all  business  systems  expected  to  be  part  of  the  target 
architecture  and  each  system’s  risks  or  challenges  to  integration  are 
explained  in  presentations  provided  to  the  Defense  Business  Council. 
It  also  stated  that  the  listing  of  systems  that  will  be  part  of  the  target 
defense  business  systems  computing  environment  and  strategy  for 
making  modifications  to  systems  that  will  be  needed  to  ensure  that 
they  comply  with  the  defense  BEA,  including  time-phased  milestones, 
performance  measures,  and  financial  resource  needs,  is  being  met  by 
the  precertification  process  and  through  the  defense  acquisition 
milestones  and  performance  measures.  Further,  it  noted  that 
information  about  how  systems  are  to  be  sequenced  according  to, 
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among  other  things,  dependencies  among  investments  is  captured  in 
presentations  and  supporting  materials  provided  to  the  Defense 
Business  Council. 

However,  while  noting  these  actions,  the  Office  of  the  Deputy  Chief 
Management  Officer  did  not  provide  evidence  that  such  information  is 
presented  during  the  precertification  process  or  to  the  Defense 
Business  Council  for  every  defense  business  system.  With  respect  to 
the  Defense  Acquisition  System,  while  the  acquisition  process  might 
support  and  be  informed  by  the  development  of  an  enterprise 
transition  plan,  the  existence  of  the  acquisition  process  does  not 
invalidate  the  need  for  the  plan  to  include  the  information  called  for  by 
the  act.  Regardless  of  whether  such  information  is  presented  to  the 
Defense  Business  Council  or  addressed  through  other  DOD 
processes,  such  as  the  department’s  acquisition  process,  the  act  still 
calls  for  the  information  to  be  included  in  the  transition  plan.  Doing  so 
will  help  ensure  informed  oversight  of  DOD  business  systems 
investments  and  help  provide  the  department  with  a  sufficient  basis 
for  sequencing  business  capabilities  and  system  functions  to  be 
subsumed  by  a  modern  system. 

DOD  also  referred  to  actions  the  department  has  taken  to  improve  its 
approach  to  managing  business  systems  investments,  including 
establishing  priorities  in  its  Strategic  Management  Plan  and 
developing  its  functional  strategies  and  organizational  execution 
plans,  which  are  discussed  in  our  report.  In  addition,  the  department 
referred  to  its  acquisition  and  budgeting  processes,  which  it  says 
provides  DOD  with  greater  visibility  into  the  business  enterprise  than 
ever  before.  While  we  recognize  that  these  actions  and  processes  are 
important  parts  of  the  department’s  efforts  to  manage  its  business 
systems  investments,  our  recommendation  focuses  on  the  need  for 
DOD  to  describe  by  when  and  how  gaps  that  we  identified  in  the 
report  would  be  included  in  its  enterprise  transition  plan.  Doing  so 
would  promote  visibility  across  the  existing  DOD  processes  and 
provide  a  basis  for  coordinating  modernization  activities  and  the 
concurrent  development  of  business  systems  in  a  manner  that  limits 
unnecessary  duplication  and  increases  the  likelihood  that  systems  will 
be  interoperable.  Consistent  with  the  department’s  data-driven 
approach  to  managing  its  business  systems,  the  approach  described 
by  officials  from  the  Office  of  the  Deputy  Chief  Management  Officer  to 
addressing  its  transition  plan  gaps  is  to  leverage  information  that  may 
already  exist  across  the  department.  Accordingly,  if  existing 
processes  already  collect  data  that  are  required  to  be  included  in  the 
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plan,  then  the  department  should  leverage  and  incorporate  these  data 
in  the  plan.  Such  an  approach  would  provide  DOD  with  an  improved 
basis  for  sequencing  business  capabilities  and  system  functions  to  be 
subsumed  by  modern  systems  while  limiting  unnecessary  duplication. 

Further,  with  regard  to  including  time-phased  end  dates  associated 
with  terminating  legacy  systems  in  phases,  DOD  stated  that  this 
requirement  is  being  met  by  the  establishment  of  a  category  of 
“legacy”  investments  that  are  to  be  retired  within  the  next  36  months 
along  with  an  explanation  of  what,  if  any,  systems  will  continue  to 
support  the  business  need.  The  department  also  stated  that  all  other 
defense  business  systems  are  considered  core  systems  and, 
therefore,  must  demonstrate  compliance  with  the  BEA  while  ensuring 
appropriate  business  process  reengineering.  Accordingly,  the 
department  stated  that  it  does  not  require  that  its  legacy  systems 
demonstrate  compliance  with  the  BEA.  While  we  agree  that  it  may  not 
be  an  efficient  use  of  resources  to  modify  legacy  systems  that  will 
soon  be  retired,  the  act  and  our  recommendation  focuses  on  the 
importance  for  all  the  other  defense  business  systems  (including  core 
systems  since  they  will  be  part  of  the  target  environment)  to  be 
included  in  the  enterprise  transition  plan  and  taking  a  phased 
approach  to  eliminating  systems.  As  such,  information  about  how 
systems  will  be  retired  in  phases  and  associated  milestones  remains 
an  important  part  of  the  transition  plan.  Thus,  we  stand  by  our 
recommendation,  but  have  added  additional  clarification  to  the 
recommendation  and  to  our  discussion  of  this  matter  in  the  body  of 
the  report  to  reflect  DOD’s  approach  for  addressing  legacy  systems. 

•  The  department  concurred  with  our  third  recommendation  to  ensure 
that  the  functional  strategies  include  all  of  the  critical  elements 
identified  in  DOD  investment  management  guidance. 

•  The  department  did  not  concur  with  our  fourth  recommendation  to 
select  and  control  its  mix  of  investments  in  a  manner  that  best 
supports  mission  needs  by  (a)  documenting  a  process  for  evaluating 
portfolio  performance  that  includes  the  use  of  actual  versus  expected 
performance  data  and  predetermined  thresholds;  (b)  ensuring  that 
portfolio  assessments  are  conducted  in  key  areas  identified  in  our  IT 
investment  management  framework:  benefits  attained;  current 
schedule;  accuracy  of  project  reporting;  and  risks  that  have  been 
mitigated,  eliminated,  or  accepted  to  date;  and  (c)  ensuring  that  the 
documents  provided  to  the  Defense  Business  Council  as  part  of  the 
investment  management  process  include  critical  information  for 
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conducting  all  assessments.  The  department  noted  that  our 
recommendation  did  not  recognize  the  significant  improvements  the 
department  made  in  2012  to  fundamentally  change  how  it  reviews  its 
defense  business  systems. 

With  regard  to  the  first  part  of  the  recommendation  about  process 
documentation,  DOD  stated  that  its  investment  review  guidance 
defines  the  review  criteria  used  to  evaluate  and  certify  business 
systems  via  organizational  execution  plans  and  was  updated  based 
on  lessons  learned  in  2012.  It  also  stated  that  the  department  intends 
to  withhold  certification  for  business  system  investments  that  are  not 
aligned  to  the  functional  strategies,  unless  a  business  system  is 
required  for  critical  operational  support,  and  that  its  IRB  does  not 
intend  to  review  acquisition  issues  associated  with  individual  business 
systems  since  that  function  is  performed  by  the  Defense  Acquisition 
System.  Our  report  acknowledged  that  DOD’s  2012  investment 
management  guidance  defines  review  criteria  that  the  Defense 
Business  Council  is  to  use  for  evaluating  and  certifying  portfolios  of 
business  systems  rather  than  individual  systems  and  credits  the 
department  for  revising  its  guidance  for  the  2014  certification  of 
defense  business  systems.  However,  the  department  has  not 
specified  a  process  for  using  the  criteria  to  make  portfolio-based 
certification  decisions  that  incorporate  the  use  of  actual  versus 
expected  performance  data  and  predetermined  thresholds,  which, 
according  to  our  IT  investment  management  framework,  is  a  major 
factor  in  evaluating  portfolio  performance  and  defining  remedial 
actions  for  underperforming  investments. 

With  regard  to  the  second  part  of  our  recommendation  covering 
portfolio  assessments,  DOD  stated  that  key  components  of  GAO’s  IT 
investment  management  framework  are  being  leveraged  to  evaluate 
the  investment  portfolios  presented  to  the  Defense  Business  Council 
and  noted  that  the  portfolio-based  review  process  is  intended  to 
complement  rather  than  duplicate  DOD’s  Defense  Acquisition  System. 
According  to  the  department,  the  Defense  Acquisition  System  is  the 
core  process  used  by  the  department  to  manage  defense  acquisitions 
and  it  already  contains  significant  elements  of  GAO’s  framework.  We 
agree  and  acknowledged  in  our  report  that  components  of  our 
framework  are  being  leveraged  in  evaluating  investment  portfolios. 
However,  the  intent  of  our  recommendation  is  not  to  duplicate  already 
existing  processes,  but  rather,  to  ensure  that  assessments  in  four  key 
areas  identified  in  our  IT  investment  management  framework  (benefits 
attained,  current  schedule,  accuracy  of  project  reporting,  and  risks 
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that  have  been  mitigated,  eliminated,  or  accepted  to  date)  are 
incorporated  and  considered  when  making  annual  certification 
decisions.  As  discussed  in  our  report,  some  investments  may  exceed 
performance  expectations  (e.g.,  delivering  at  lower  cost,  in  less  time, 
and  with  better  benefits  than  expected)  and  in  these  cases,  the  board 
may  wish  to  accelerate  an  investment’s  funding  or  schedule, 
reallocate  resources  within  the  overall  portfolio,  or  make  some  other 
type  of  adjustment. 

With  regard  to  the  third  part  of  our  recommendation  on  documentation 
for  conducting  assessments,  the  department  stated  that  the 
information  derived  for  strategic  alignment,  cost,  utility,  and 
compliance  is  already  given  to  the  Defense  Business  Council  and 
incorporates  objective  information  derived  from  authoritative 
documents  and  data  sources.  It  also  noted  that  the  documentation  is 
supported  by  input  from  the  functional  area  owners  and  subject  matter 
experts.  However,  as  discussed  in  our  report,  the  department’s 
investment  management  guidance  does  not  call  for  the  organizational 
execution  plans,  which  are  the  key  documents  used  by  the  Defense 
Business  Council  to  make  certification  decisions,  to  include  critical 
information  for  conducting  assessments  associated  with  strategic 
alignment,  utility,  and  total  cost  criteria. 

Notwithstanding  the  actions  DOD  has  taken  in  these  areas,  we 
continue  to  believe  our  recommendation,  in  its  entirety,  is  warranted. 

•  With  regard  to  our  fifth  recommendation,  the  department  partially 
concurred.  Specifically,  DOD  agreed  with  the  first  two  parts  of  our 
recommendation  that  the  BEA  and  business  process  reengineering 
compliance  assessments  be  used  to  support  organizational 
transformation  efforts  by  disclosing  relevant  information  about  known 
weaknesses  and  identifying  milestones  for  completing  selected 
validations.  However,  it  did  not  agree  with  the  third  part  of  the 
recommendation  on  ensuring  that  appropriate  business  process 
reengineering  assertions  have  been  completed  for  all  investments 
submitted  for  the  fiscal  year  2014  certification  reviews  prior  to  the 
certification  of  funds.  The  department  noted  that  requiring  this 
assertion  for  systems  solely  in  sustainment  or  scheduled  for 
retirement  would  divert  scarce  resources  to  tasks  that  would  provide 
no  benefit.  Further,  it  stated  that  its  latest  Business  Process 
Reengineering  Assessment  Guidance  of  September  28,  2012, 
provided  DOD  with  a  business  process  reengineering  standard,  a  plan 
for  validating  a  sample  of  defense  business  programs'  compliance 
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assertions,  and  a  requirement  for  precertification  authorities  to 
develop  a  Plan  of  Action  for  all  defense  programs  that  did  not  assert 
compliance  in  fiscal  year  2013  to  become  compliant,  which  was  also 
documented  in  related  investment  decision  memoranda. 

We  have  recognized  all  of  these  steps  cited  by  the  department  in  our 
report.  Nonetheless,  we  stand  by  our  recommendation  for  completing 
appropriate  business  process  reengineering  assertions  for  all 
investments,  which  is  a  requirement  of  the  NDAA  and  in  accordance 
with  DOD  guidance.  Specifically,  DOD's  Business  Process 
Reengineering  Assessment  Guidance  describes  how  the  standard 
DOD  has  developed  should  be  applied  to  varying  degrees  depending 
on  the  life-cycle  stage  of  each  defense  business  system,  providing  a 
reduced  burden  for  systems  in  sustainment  and  eliminating  the 
requirement  for  those  systems  designated  as  legacy,  which  for  fiscal 
year  2013  included  about  99  systems.  We  acknowledge  DOD’s 
approach  as  described  in  its  guidance,  but  maintain  that  a 
precertification  authority  would  still  need  to  make  the  determination 
that  appropriate  business  process  reengineering  efforts  had  been 
undertaken.  The  precertification  authority  would  also  be  able  to  assert 
and  validate  compliance  for  a  system  in  sustainment  if  the  system  has 
followed  DOD  guidance  and  for  a  legacy  system  scheduled  to  be 
retired  within  36  months  because  no  reengineering  was  required. 
However,  for  the  fiscal  year  2013  reviews,  the  use  of  the  action  items 
requiring  precertification  authorities  to  develop  an  action  plan  for  all 
defense  business  programs,  which  did  not  assert  business  process 
reengineering  compliance,  resulted  in  more  than  half  of  the  covered 
business  systems  being  asserted  as  non-compliant  at  the  time  they 
were  certified.  We  also  acknowledge  that  the  department  is  working  to 
remedy  this  situation  for  fiscal  year  2014,  but  our  recommendation 
has  been  made  to  ensure  that  asserting  compliance,  as  required  by 
the  act,  is  a  departmentwide  priority.  Thus,  we  continue  to  believe  our 
recommendation  is  warranted. 

•  The  department  did  not  concur  with  our  sixth  recommendation  to 
develop  a  skills  inventory,  needs  assessment,  gap  analysis,  and  plan 
to  address  identified  gaps  as  part  of  a  strategic  approach  to  human 
capital  planning.  In  particular,  DOD  stated  that  we  did  not  recognize 
that  the  Office  of  the  Deputy  Chief  Management  Officer  is  a  relatively 
small  component  of  the  Office  of  the  Secretary  of  Defense  and  that 
we  had  directed  the  office  to  undertake  a  level  of  activity  normally 
designed  for  an  entire  agency.  While  we  recognize  that  the  office  is  a 
relatively  small  component  of  the  overall  department,  as  we  reported, 
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the  success  of  any  program  depends  on  effectively  leveraging  people 
to  achieve  defined  outcomes  and  results.  Further,  without  knowing  if  it 
has  the  right  number  of  staff  required  and  the  specific  skills  and 
abilities  needed,  the  office  will  be  challenged  in  its  ability  to  effectively 
achieve  its  mission  of  leading  and  enabling  end-to-end  integration  and 
improvement  of  business  operations  for  approximately  1 ,200  business 
system  investments — totaling  about  $6.8  billion  in  approved  funding 
for  fiscal  year  2013. 

DOD  also  stated  that  the  report  misunderstands  statements  made  by 
officials  within  the  office  and  draws  conclusions  about  the  impact  of 
staffing  levels  that  are  not  supported  by  the  report’s  findings.  More 
specifically,  the  department  stated  that  the  office  thoroughly 
considered  the  skills  of  its  employees,  sought  to  optimally  align  them 
organizationally,  and  has  used  hiring  actions  to  bring  additional  skill 
sets  onboard  that  filled  identified  gaps.  While  our  report 
acknowledged  the  efforts  the  office  has  taken  to  conduct  analyses  of 
the  functions  that  it  is  to  perform,  we  also  note  that  the  office  did  not 
provide  evidence  that  it  has  used  a  documented,  fact-based,  data- 
driven,  methodology  to  assess  human  capital  needs  and  existing 
capabilities,  or  performed  a  gap  analysis  of  the  number  of  staff 
required  and  the  specific  skills  and  abilities  needed  to  effectively 
achieve  its  mission. 

The  department  further  noted  that  the  report  attributes  consequences 
that  are  beyond  what  is  supported  by  the  report’s  findings  and  that 
weaknesses  we  identified  in  the  department’s  federated  architecture, 
investment  management  data,  and  transition  plan  content  are 
departmental  weaknesses  and  are  not  simply  due  to  the  staffing 
levels  of  the  Office  of  the  Deputy  Chief  Management  Officer.  While  we 
recognize  DOD’s  “tiered  accountability”  approach  to  business  systems 
modernization,  where  responsibility  and  accountability  for  business 
architectures  and  systems  investment  management  are  assigned  to 
different  levels  in  the  department,  we  do  not  agree  with  the 
department’s  assertions.  Our  report  does  not  link  the  weaknesses  to 
the  staffing  levels  but  rather  it  states  that  the  efforts  of  the  Office  of 
the  Deputy  Chief  Management  Officer  have  been  impacted  by  the 
lack  of  a  strategic  approach  to  managing  its  human  capital  needs. 
Given  the  level  of  oversight  entrusted  to  the  office  relative  to  business 
systems  modernization,  it  is  important  that  it  address  the  weaknesses 
identified  in  our  report.  Thus,  it  is  imperative  for  the  office  to  help 
ensure  that  its  workforce  is  appropriately  sized  and  has  the  right  mix 
of  knowledge,  skills,  and  training.  Adopting  a  more  strategic  and 
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proactive  approach  to  managing  its  human  capital  will  be  a  positive 
step  forward  in  helping  to  ensure  that  such  weaknesses  will  be 
addressed  in  a  timely  and  cost-effective  manner.  As  such,  we  stand 
by  this  recommendation. 

•  The  department  concurred  with  our  seventh  recommendation  to 
ensure  that  complete  documentation,  such  as  root  cause  analyses, 
assessments  of  existing  interfaces  for  reuse  opportunities,  and 
performance  metrics  related  to  the  reengineering  efforts,  are  provided 
as  part  of  the  fiscal  year  2014  certification  and  approval  process  for 
the  IPPS-A,  IPPS-N,  AF-IPPS,  and  iEHR  investments.  It  stated  that  its 
Business  Process  Reengineering  Assessment  Guidance  of 
September  28,  2012,  addressed  our  concerns  in  its  standard 
questions.  We  support  the  department’s  efforts  to  address  our 
recommendation  and  reiterate  the  importance  of  ensuring  that 
complete  documentation,  in  support  of  the  standard  questions,  is 
provided  as  part  of  the  certification  and  approval  process. 

•  The  department  did  not  concur  with  our  eighth  recommendation  to 
determine  whether  funds  were  properly  obligated  under  10  U.S.C. 
§2222(a)-(b)  for  systems  for  which  appropriate  business  process 
reengineering  assertions  were  not  completed.  DOD  stated  that  it 
believes  that  it  has  already  met  the  conditions  of  the  act  through 
reviews  by  the  precertification  authorities  and  the  Defense  Business 
Council.  The  department  noted,  as  an  example,  that  it  considers  any 
existing  defense  business  systems  with  a  retirement  date  within  36 
months  of  certification  to  be  legacy  systems,  so  development  and 
modernization  funds  for  these  legacy  systems  have  been  removed 
from  the  budget.  It  further  noted  that  the  requirement  to  retire  these 
systems  is  included  in  all  relevant  documentation  generated;  is 
approved  by  the  Defense  Business  Council  and  the  Defense  Business 
Systems  Management  Committee;  and  therefore  has  been  reviewed 
for  compliance.  DOD  stated  that  its  approach  to  legacy  systems 
scheduled  for  retirement  conserves  resources  of  the  department.  The 
department  also  stated  that  all  other  defense  business  systems 
(including  existing  systems  that  will  be  modernized)  are  considered 
core  systems  and,  therefore,  must  demonstrate  compliance  to  the 
BEA  while  ensuring  appropriate  business  process  reengineering. 
Further,  the  department  stated  that  its  revised  Business  Process 
Reengineering  Assessment  Guidance  memorandum  of  September 
28,  2012,  specifically  addressed  the  requirement  for  the 
precertification  authority  to  develop  a  Plan  of  Action  for  all  defense 
business  programs  that  did  not  assert  business  process  reengineering 
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compliance  in  fiscal  year  201 3  to  become  compliant;  this  requirement 
was  also  documented  in  the  investment  decision  memorandum  for 
each  organizational  execution  plan  as  a  condition  to  the  fiscal  year 
2013  certification  of  funds. 

Consistent  with  a  determination  that  appropriate  business  process 
reengineering  efforts  have  been  undertaken,  we  acknowledge  that  a 
legacy  system  scheduled  to  be  retired  within  36  months  would  be 
asserted  in  compliance  because  no  reengineering  was  required. 
However,  for  all  other  defense  business  systems  that  are  not  legacy 
systems  scheduled  for  retirement,  we  do  not  agree  that  the 
department  has  already  met  the  conditions  of  the  act  through  reviews 
by  the  precertification  authorities  and  the  Defense  Business  Council. 
We  maintain  that,  consistent  with  the  act,  a  precertification  authority 
would  still  need  to  make  the  determination  that  appropriate  business 
process  reengineering  efforts  have  been  undertaken  as  a  condition  for 
obligating  funds  for  covered  defense  business  system  programs.  We 
acknowledge  that  DOD  has  some  discretion  in  deciding  what 
constitutes  the  undertaking  of  appropriate  business  process 
reengineering  efforts.  Further,  the  act  provides  alternative  bases  for 
certification  prior  to  the  obligation  of  funds  that  DOD  could  consider  in 
some  situations.  However,  the  majority  of  systems  did  not  assert 
business  process  reengineering  compliance  at  the  time  they  were 
certified  and  DOD’s  use  of  a  Plan  of  Action  in  lieu  of  determining  that 
appropriate  business  process  reengineering  efforts  have  been 
undertaken  does  not  directly  address  our  recommendation  to 
determine  whether  funds  were  properly  obligated  under  the  act. 
Indeed,  DOD’s  description  of  its  actions  actually  highlights  the 
apparent  absence  of  compliance  with  the  condition  on  the  certification 
of  funds  for  the  majority  of  the  systems.  In  this  regard,  the  subsequent 
higher-level  review  and  certification  of  these  systems  does  not  rectify 
the  absence  of  the  necessary  determinations  by  the  precertification 
authorities,  as  required  by  the  act.  Accordingly,  we  reassert  our 
recommendation. 


We  are  sending  copies  of  this  report  to  the  appropriate  congressional 
committees;  the  Director,  Office  of  Management  and  Budget;  the 
Secretary  of  Defense;  and  other  interested  parties.  This  report  also  is 
available  at  no  charge  on  the  GAO  website  at  http://www.gao.gov. 

If  you  or  your  staff  members  have  any  questions  on  matters  discussed  in 
this  report,  please  contact  me  at  (202)  512-6304  or  melvinv@gao.gov. 
Contact  points  for  our  Offices  of  Congressional  Relations  and  Public 
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Affairs  may  be  found  on  the  last  page  of  this  report.  GAO  staff  who  made 
major  contributions  to  this  report  are  listed  in  appendix  V. 


Valerie  C.  Melvin 
Director 

Information  Management  and  Technology  Resources  Issues 
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List  of  Committees 


The  Honorable  Carl  Levin 
Chairman 

The  Honorable  James  M.  Inhofe 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Dick  Durbin 
Chairman 

The  Honorable  Thad  Cochran 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Howard  P.  McKeon 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  C.W.  Bill  Young 
Chairman 

The  Honorable  Pete  Visclosky 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 


Page  61 


GAO-13-557  DOD  Business  Systems  Modernization 


Appendix  I:  Objective,  Scope,  and 
Methodology 


Our  objective  was  to  assess  the  actions  by  the  Department  of  Defense  to 
comply  with  section  332  of  the  National  Defense  Authorization  Act  for 
Fiscal  Year  2005  and  amendments.1  These  include  (1)  developing  a 
business  enterprise  architecture  (BEA)  to  cover  all  defense  business 
systems,  (2)  developing  a  transition  plan  for  implementing  the 
architecture,  (3)  identifying  systems  information  in  its  annual  budget 
submission,  (4)  establishing  a  system  investment  approval  and 
accountability  structure  along  with  an  investment  review  process,  and  (5) 
certifying  and  approving  any  business  system  program  costing  in  excess 
of  $1  million.  In  addition,  we  reviewed  the  department’s  efforts  to  develop 
a  strategic  approach  to  human  capital  management  and  address  our  prior 
open  recommendations. 

To  address  the  provision  associated  with  developing  a  business 
architecture,  we  analyzed  version  10.0  of  the  BEA,  which  was  released  in 
February  2013  relative  to  the  act’s  architectural  requirements,  DOD’s 
September  201 1  Strategic  Management  Plan,  and  recommendations  that 
our  previous  annual  reports  in  response  to  the  act  identified  as  not  being 
fully  implemented.  Specifically,  we  interviewed  officials  from  the  Office  of 
the  Deputy  Chief  Management  Officer  and  reviewed  written  responses 
and  related  documentation  on  steps  completed,  under  way,  or  planned  to 
address  the  act’s  requirements  and  previously  reported  weaknesses.  We 
then  reviewed  architectural  artifacts  in  BEA  10.0  to  validate  the  responses 
and  identify  any  discrepancies.  In  addition,  we  reviewed  documentation 
and  interviewed  officials  from  the  Office  of  the  Deputy  Chief  Management 
Officer  and  the  military  departments  about  efforts  to  establish  a  federated 
business  mission  area  enterprise  architecture. 

To  address  the  provision  associated  with  developing  a  transition  plan,  we 
analyzed  DOD’s  enterprise  transition  plan,  released  in  December  2012, 
relative  to  the  act’s  transition  plan  requirements  and  related  findings 
documented  in  previous  GAO  reports.  Specifically,  we  interviewed 
officials  from  the  Office  of  the  Deputy  Chief  Management  Officer  and 
reviewed  written  responses  and  related  documentation  on  steps 
completed,  under  way,  or  planned  to  address  the  act’s  requirements  and 


Vub.  L.  No.  108-375,  §  332,  118  Stat.  1811,  1851-1856  (2004),  as  amended  by  the 
NDAA  for  Fiscal  Year  2013,  Pub.  L.  No.  112-239,  126  Stat.  1632  (2012),  the  NDAA  for 
Fiscal  Year  2012,  Pub.  L.  No.  112-81,  125  Stat.  1298  (2011),  and  the  NDAA  for  Fiscal 
Year  2010,  Pub.  L.  No.  111-84,  123  Stat.  2190  (2009)  (codified  in  part  at  10  U.S.C.  § 
2222.) 
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Appendix  I:  Objective,  Scope,  and 
Methodology 


previously  reported  weaknesses.  We  then  reviewed  enterprise  transition 
plan  documentation  (e.g.,  organizational  execution  plans  and  Office  of 
Management  and  Budget  (OMB  exhibit  300s))  to  validate  the  responses 
and  identify  any  discrepancies. 

To  assess  the  system  information  in  DOD’s  fiscal  year  2013  information 
technology  budget  submission,  we  analyzed  and  compared  information 
contained  in  the  department’s  system  that  is  used  to  prepare  its  budget 
submission — Select  and  Native  Programming  Data  Input  System — 
Information  Technology  (SNAP-IT)  with  information  in  the  department’s 
authoritative  business  systems  inventory — Defense  Information 
Technology  Portfolio  Repository  (DITPR).  We  were  unable  to  determine 
whether  DOD’s  fiscal  year  2014  information  technology  budget 
submission  was  prepared  in  accordance  with  the  criteria  set  forth  in  the 
act  because  the  budget  submission  was  not  released  in  time  for  us  to 
review  it  for  this  report.  We  also  interviewed  officials  from  the  Office  of  the 
Deputy  Chief  Management  Officer  and  office  of  DOD’s  Chief  Information 
Officer  to  discuss  the  accuracy  and  comprehensiveness  of  information 
contained  in  the  DITPR  system,  the  discrepancies  in  the  information 
contained  in  the  DITPR  and  SNAP-IT  systems,  and  efforts  under  way  or 
planned  to  address  these  discrepancies  and  our  prior  recommendations. 

To  address  the  act’s  provision  for  establishing  an  investment  approval 
and  accountability  review  structure  along  with  an  investment  review 
process,  we  reviewed  and  analyzed  the  department’s  current  policies, 
procedures,  and  guidance  relative  to  the  act’s  requirements  and  criteria 
documented  in  our  IT  Investment  Management  framework.2  Specifically, 
to  determine  whether  a  single  Investment  Review  Board,  composed  of 
appropriate  senior  executives,  had  been  established,  we  reviewed  the 
October  2012  Defense  Business  Council  charter.  To  determine  whether 
the  foundation  for  DOD’s  new  investment  management  process  had  been 
fully  established,  we  reviewed  and  analyzed  the  six  functional  strategies 
prepared  for  the  fiscal  year  2013  certification  cycle  on  the  extent  to  which 
they  included  the  critical  elements  called  for  in  the  department’s  June 


2GAO’s  HIM  framework  provides  a  method  for  evaluating  and  assessing  how  well  an 
agency  is  selecting  and  managing  its  IT  resources.  The  framework,  which  describes  five 
progressive  stages  of  maturity  that  an  agency  can  achieve  in  its  investment  management 
capabilities,  was  developed  on  the  basis  of  our  research  into  the  IT  investment 
management  practices  of  leading  private-  and  public-sector  organizations.  See  GAO, 
Information  Technology  Investment  Management:  A  Framework  for  Assessing  and 
Improving  Process  Maturity,  GAO-04-394G  (Washington,  D.C.:  March  2004). 
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2012  investment  management  guidance.  We  also  reviewed  and  analyzed 
the  investment  management  guidance  to  determine  whether  criteria  and 
procedures  had  been  fully  defined  for  making  portfolio-based  investment 
decisions  and  the  extent  to  which  they  aligned  to  our  IT  investment 
management  framework.  In  addition,  we  reviewed  and  analyzed  the  April 

2013  revised  investment  management  guidance  and  updated  versions  of 
the  six  functional  strategies  prepared  for  the  fiscal  year  2014  certification 
cycle  to  determine  whether  improvements  had  been  made.  Finally,  we 
interviewed  cognizant  officials  in  the  Office  of  the  Deputy  Chief 
Management  Officer  to  gain  a  better  understanding  of  the  process  and  its 
outputs  and  planned  improvements  for  fiscal  year  2014  defense  business 
system  certifications. 

To  determine  whether  the  department  was  certifying  and  approving 
business  system  programs  costing  in  excess  of  $1  million  in  accordance 
with  the  act’s  provisions,  we  reviewed  and  analyzed  DOD’s  fiscal  year 
2013  report  to  Congress,  all  organizational  execution  plans  and  their 
respective  precertification  request  memoranda,  and  associated  portfolio 
attachments  submitted  as  part  of  the  fiscal  year  2013  investment  review 
process.  This  also  included  reviewing  and  analyzing  investment  decision 
memoranda  which  documented  the  decisions  made  by  the  Defense 
Business  Council.  We  also  compared  our  results  with  the  information 
reported  in  DOD’s  annual  report  to  Congress  to  identify  any  discrepancies 
and  discussed  these  with  cognizant  officials  from  the  Office  of  the  Deputy 
Chief  Management  Officer.  Due  to  DOD’s  manual  data  cleanup  efforts 
during  the  fiscal  year  2013  investment  review  process,  we  were  unable  to 
verify  the  exact  number  and  value  of  certification  requests. 

As  part  of  our  evaluation  under  this  requirement,  we  also  sought  to 
identify  the  process  associated  with  asserting  BEA  and  business  process 
reengineering  compliance  for  covered  defense  business  systems  and  the 
documentation  used  to  support  those  assertions.  To  accomplish  this,  we 
reviewed  DOD  guidance  and  related  documentation  that  applied  to  the 
fiscal  year  2013  investment  portfolio  reviews  and  relevant  updates  to  that 
guidance  issued  during  our  audit.  We  then  applied  the  guidance  to  four 
investments  that  we  selected — Air  Force  Integrated  Personnel  and  Pay 
System,  Integrated  Personnel  and  Pay  System-Navy,  Integrated 
Personnel  and  Pay  System-Army,  and  the  Integrated  Electronic  Health 
Record  System — based  on  the  following  criteria:  the  investment  was  (1) 
one  of  the  top  25  largest  dollar  business  system  programs  identified  using 
the  2013  IT  systems  budget  submission,  (2)  not  the  recent  subject  of 
DOD  Inspector  General  review  that  included  review  of  business  process 
reengineering  activities,  and  (3)  an  example  from  one  of  the  military 
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departments  or  defense  agency  or  activity.  In  reviewing  these 
investments,  we  analyzed  and  compared  BEA  and  business  process 
reengineering  supporting  documentation  that  was  submitted  as  part  of 
compliance  assertions  for  fiscal  year  2013  request  for  certification  of 
funds,  to  requirements  in  DOD’s  201 1  BEA  compliance  guidance,3  DOD’s 
business  process  reengineering  assessment  guidance,4  and  our 
Business  Process  Reengineering  Assessment  Guide.5  The  supporting 
documentation,  among  other  things,  documented  the  current  and  target 
processes  and  described  business  performance  metrics.  We  also 
interviewed  officials  from  each  system’s  component — the  Departments  of 
Air  Force,  Army,  Navy,  and  the  Tricare  Management  Activity — and 
discussed  the  program-specific  documentation  submissions  with  program 
officials.  In  addition,  we  interviewed  officials  from  the  Office  of  the  Deputy 
Chief  Management  Officer  about  steps  completed,  under  way,  or  planned 
relative  to  BEA  and  business  process  reengineering  compliance  and 
validation  activities.  Further,  we  interviewed  DOD  officials  about  efforts  to 
understand,  track,  and  report  outcomes  related  to  business  process 
reengineering  efforts.  We  also  reviewed  DOD’s  annual  report  to  Congress 
to  determine  if  business  process  reengineering  outcomes  had  been 
reported. 

To  determine  the  extent  to  which  the  Office  of  the  Deputy  Chief 
Management  Officer  had  implemented  a  strategic  approach  to  human 
capital  management,  we  reviewed  our  own  guidance  and  reports  on 
federal  agencies’  workforce  planning  and  human  capital  management 
efforts.6  We  then  met  with  officials  from  the  Office  of  the  Deputy  Chief 
Management  Officer,  including  the  Assistant  Deputy  Chief  Management 


department  of  Defense,  DOD  Business  Enterprise  Architecture:  Compliance  Guidance 
for  BEA  8.0  (Mar.  11, 2011). 

department  of  Defense,  Guidance  for  the  Implementation  of  Section  1072-  Business 
Process  Reengineering  (Apr.  30,  201 1 );  and  Business  Process  Reengineering 
Assessment  Guidance  (Sept.  28.  2012). 

5GAO,  Business  Process  Reengineering  Assessment  Guide  (Version  3), 
GAO/AIMD-IO.1.15  (Washington,  D.C.:  May  1997). 

6  GAO,  National  Archives  and  Records  Administration:  Oversight  and  Management 
Improvements  Initiated,  but  More  Action  Needed,  GAO-1 1-15  (Washington,  D.C.:  Oct  5, 
2010);  Information  Technology:  FBI  Has  Largely  Staffed  Key  Modernization  Program,  but 
Strategic  Approach  to  Managing  Program’s  Human  Capital  Is  Needed,  GAO-07- 19 
(Washington,  D.C.:  Oct.  16,  2006);  and  A  Model  of  Strategic  Human  Capital  Management, 
GAO-02-373SP  (Washington,  D.C.:  Mar.  15,  2002). 
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Officer  to  discuss  the  office’s  human  capital  planning  efforts  and  analyzed 
related  documentation  provided  by  the  office  on  steps  completed,  under 
way,  or  planned  to  address  strategic  human  capital  management 
activities  discussed  in  our  guidance. 

To  determine  the  extent  to  which  DOD  had  implemented  our  prior 
recommendations,  we  identified  48  recommendations  related  to  DOD’s 
business  systems  modernization  efforts  that  we  made  in  our  reports  from 
2005  to  2008  and  summarized  the  implementation  status  of  each.  For 
those  recommendations  that  had  not  been  implemented  after  more  than  4 
years  since  we  issued  the  associated  report,  we  considered  them  closed 
and  not  implemented.  We  then  focused  on  the  15  recommendations 
related  to  DOD’s  business  systems  modernization  efforts  that  we  made  in 
our  reports  from  2009  to  2013.  In  reviewing  these  15  recommendations, 
we  obtained  and  analyzed  documentation  relative  to  corrective  actions 
taken  and  planned.  Documentation  that  we  reviewed  included  DOD’s 
2012  vision  for  the  BEA,7  the  most  recent  version  of  DOD’s  BEA 
compliance  guidance,  and  DOD’s  2012  and  2013  investment 
management  process  guidance.  Further,  we  reviewed  documentation 
associated  with  changes  to  the  Office  of  the  Deputy  Chief  Management 
Officer’s  staffing  levels,  as  well  as  the  department’s  enterprise  transition 
plans  for  fiscal  years  201 1 , 2012,  and  2013,  and  its  fiscal  year  2013 
report  to  Congress. 

We  conducted  this  performance  audit  at  DOD  and  military  department 
offices  from  October  2012  to  May  2013,  in  accordance  with  generally 
accepted  government  auditing  standards.  Those  standards  require  that 
we  plan  and  perform  the  audit  to  obtain  sufficient,  appropriate  evidence  to 
provide  a  reasonable  basis  for  our  findings  and  conclusions  based  on  our 
audit  objectives.  We  believe  that  the  evidence  obtained  provides  a 
reasonable  basis  for  our  findings  and  conclusions  based  on  our  audit 
objective. 


department  of  Defense,  Same  Mission ,  New  Vision.  The  Future  of  the  DOD  Business 
Enterprise  Architecture  (March  2012). 
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The  department’s  BEA  focuses  on  documenting  information  associated 
with  its  15  end-to-end  business  process  areas.  In  particular,  the 
department’s  Strategic  Management  Plan  has  identified  the  Hire-to-Retire 
and  Procure-to-Pay  business  process  areas  as  its  priorities.  According  to 
the  department,  the  process  of  documenting  the  needed  architecture 
information  also  includes  working  to  refine  and  streamline  each  of  the 
associated  end-to-end  business  processes. 


Business  process 

Description 

Acquire-to-Retire 

Obtain,  manage,  and  dispose  of  accountable  and  reportable  property  (capitalized  and 
noncapitalized  assets)  through  their  entire  life  cycle. 

Budget-to-Report 

Plan,  formulate,  create,  execute  against,  and  report  on  the  budget  and  business  activities  of  the 
entity. 

Concept-to-Product 

Identify  product  needs,  and  plan  and  execute  all  necessary  activities  to  bring  a  product  from  initial 
concept  to  full  production. 

Cost  Management 

Identify,  collect,  measure,  accumulate,  analyze,  interpret,  and  communicate  cost  information  to 
accomplish  the  many  objectives  associated  with  control,  decision  making,  planning,  and  reporting. 

Deployment-to- 

Redeployment/Retrograde 

Plan,  notify,  deploy,  sustain,  recall,  and  reset  tactical  units  to  and  from  theaters  of  engagement. 

Environmental  Liabilities 

Identify  environmental  cleanup,  closure,  or  disposal  issues  that  represent  an  environmental  liability 
of  the  department,  to  develop  cost  estimates  and  expenditures  related  to  the  actions  required  to 
eliminate  an  identified  environmental  liability,  and  to  report  appropriate  financial  information  about 
the  environmental  liability. 

Hire-to-Retire 

Plan  for,  hire,  classify,  develop,  assign,  track,  account  for,  compensate,  retain,  and  separate  the 
persons  needed  to  accomplish  aspects  of  the  DOD  mission. 

Market-to-Prospect 

Establish  marketing  plans,  identify  target  markets,  plan  and  define  marketing  campaigns,  execute 
marketing  campaigns,  and  measure  and  evaluate  the  performance  of  marketing  campaigns. 

Order-to-Cash 

Accept  and  process  customer  orders  for  services  and/or  inventory  held  for  sale. 

Plan-to-Stock 

Plan,  procure,  produce,  inventory,  and  stock  materials  used  both  in  operations  and  maintenance  as 
well  as  for  sale. 

Procure-to-Pay 

Obtain  goods  and  services. 

Proposal-to-Reward 

Life  cycle  of  the  grant  process  from  the  grantor  perspective.  It  includes  all  the  business  functions 
necessary  to  plan,  solicit,  review,  award,  perform,  monitor,  and  close  out  a  grant. 

Prospect-to-Order 

Generate  and  sustain  sales  by  pursuing  qualified  leads,  employing  effective  sales  techniques, 
efficient  order  processing,  maintaining  customer  relationships  and  providing  support  functions  to 
include  service,  personnel,  and  financial  impacts. 

Service  Request-to-Resolution 

Perform  maintenance  on  materiel/assets  requiring  repair  or  complete  rebuild  of  parts,  assemblies, 
subassemblies,  and  end-items,  including  the  manufacture  of  parts,  modifications,  testing,  and 
reclamation  as  required.  It  also  includes  the  process  whereby  buildings  and  other  fixed  facilities  are 
maintained  and  renovated  during  their  life  cycle. 

Service-to-Satisfaction 

Determine  service  requirements,  secure  funding,  contract  with  outside  vendor,  establish  service, 
and  measure  customer  satisfaction. 

Source:  GAO  based  on  DOD  documentation. 
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GAO  report  information  and  recommendation 

Implemented 

In 

Yes  Process 

GAO  assessment 

GAO-09-586:  DOD  Business  Systems 
Modernization:  Recent  Slowdown  in 
Institutionalizing  Key  Management  Controls 
Needs  to  be  Addressed,  May  1 8,  2009 

1.  To  ensure  that  DOD  continues  to  implement  the 
full  range  of  institutional  management  controls 
needed  to  address  its  business  systems 
modernization  high-risk  area,  the  Secretary  of 
Defense  should  direct  the  Deputy  Secretary  of 
Defense,  as  chair  of  the  Defense  Business 

Systems  Management  Committee  and  as  DOD’s 
Chief  Management  Officer,  to  resolve  the  issues 
surrounding  the  roles,  responsibilities,  authorities, 
and  relationships  of  the  Deputy  Chief  Management 
Officer  and  the  military  department  Chief 
Management  Officers  relative  to  the  business 
enterprise  architecture  (BEA)  and  enterprise 
transition  plan  federation  and  business  system 
investment  management. 

X 

The  department  stated  that  it  has  a  number  of  policy  and 
guidance  documents  currently  under  development  to  help 
clarify  the  roles  and  responsibilities  associated  with 
development  of  the  BEA  and  investment  management.  For 
example,  the  department’s  most  recent  investment 
management  guidance  specifies  roles  and  responsibilities 
associated  with  managing  portfolios  of  business  systems.  In 
addition,  DOD  has  established  a  forum  for  discussing  the 
development  of  the  federated  BEA  and  is  currently  working  to 
establish  a  BEA  Configuration  Control  Board  charter,  which  is 
intended  to  document  the  roles  and  responsibilities  for  the 
review  and  approval  of  proposed  BEA  content.  However,  the 
department  has  yet  to  clarify  in  policy  or  guidance  the  roles, 
responsibilities,  authorities,  and  relationships  between  the 
Deputy  Chief  Management  Officer  and  military  department 
Chief  Management  Officers  relative  to  the  BEA  and 
enterprise  transition  plan  federation. 

2.  To  ensure  that  business  system  investment 
reviews  and  related  certification  and  approval 
decisions,  as  well  as  annual  budget  submissions, 
are  based  on  complete  and  accurate  information, 
the  Secretary  of  Defense  should  direct  the 
appropriate  DOD  organizations  to  develop  and 
implement  plans  for  reconciling  and  validating  the 
completeness  and  reliability  of  information  in  its 
Defense  Information  Technology  Portfolio 

Repository  (DITPR)  and  Select  and  Native 
Programming  Data  Input  System — Information 
Technology  system  data  repositories,  and  to 
include  information  on  the  status  of  these  efforts  in 
the  department’s  fiscal  year  2010  report  in 
response  to  the  act. 

X 

According  to  DOD,  the  department  is  fully  committed  to 
integrating  the  data  in  the  system  data  repositories.  DOD 

Chief  Information  Officer  officials  stated  that  a  new  portal  that 
will  provide  an  integrated  view  of  information  from  its  two 
portals  will  reach  a  limited  operational  capability  in  May  2013. 
However,  DOD  has  yet  to  fully  implement  its  plans  and  to 
complete  the  integration  of  the  repositories  and  validate  the 
completeness  and  reliability  of  the  information. 

GAO-IO-663:  Business  Systems  Modernization: 
Scope  and  Content  ofDOD’s  Congressional 
Report  and  Executive  Oversight  of  Investments 
Need  to  Improve,  May  24,  2010 

1.  To  ensure  that  Investment  Review  Board  (IRB)  X 
certification  actions  are  better  informed  and 
justified,  the  Secretary  should  direct  the  Deputy 
Secretary  to  ensure  that  DOD  guidance  be  revised 
to  include  provisions  that  require  IRB  certification 
submissions  disclose  program  weaknesses  raised 


The  enterprise  transition  plans  for  fiscal  year  2011  and  2012 
and  system  dashboards  reflected  related  GAO  reports.  In 
addition,  the  department’s  fiscal  year  2014  investment 
management  guidance  issued  by  the  Office  of  the  Deputy 
Chief  Management  Officer  now  requires  the  precertification 
authorities  to  include  any  open  GAO  recommendations  for 
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GAO  report  information  and  recommendation 

Implemented 

In 

Yes  Process 

GAO  assessment 

by  GAO  and  the  status  of  actions  to  address  our 
recommendations  to  correct  the  weaknesses. 

program  weaknesses  as  well  as  a  status  update  on 
addressing  GAO  recommendations  as  part  of  the  certification 
requests. 

2.  To  facilitate  congressional  oversight  and  promote 
departmental  accountability,  the  Secretary  of 
Defense  should  direct  the  Deputy  Secretary  of 
Defense,  as  the  chair  of  the  Defense  Business 
Systems  Management  Committee,  to  ensure  that 
the  scope  and  content  of  future  DOD  annual 
reports  to  Congress  on  compliance  with  section 

332  of  the  National  Defense  Authorization  Act 
(NDAA)  be  expanded  to  include  (1)  cost,  capability, 

X 

While  the  department’s  fiscal  year  2013  report  to  Congress 
included  a  list  of  certification  actions,  it  did  not  include  cost, 
capability,  and  benefit  performance  measures  for  each 
business  system  investment  and  actual  performance  against 
these  measures.  According  to  officials  from  the  Office  of  the 
Deputy  Chief  Management  Officer,  the  department  elected 
not  to  seek  measures  data  for  fiscal  year  2012  given  the 
significant  changes  as  a  result  of  the  NDAA,  but  it  plans  to 
provide  measurement  data  for  the  fiscal  year  2014  cycle. 

and  benefit  performance  measures  for  each 
business  system  modernization  investment  and 
actual  performance  against  these  measures;  and 
(2)  all  certification  actions,  as  defined  in  DOD 
guidance,  which  were  taken  in  the  previous  year  by 
the  department  on  its  business  system 
modernization  investments. 


GAO-1 1-684:  Department  of  Defense:  Further 
Actions  Needed  to  Institutionalize  Key  Business 
System  Modernization  Management  Controls, 

June  29,  2011 

1.  To  address  the  uncertainty  and  pending  X 

decisions  surrounding  the  roles  and  responsibilities 
of  key  organizations,  the  Secretary  of  Defense 
should  expeditiously  complete  the  implementation 
of  the  announced  transfer  of  functions  of  the 
Business  Transformation  Agency  and  the  Office  of 
the  Assistant  Secretary  of  Defense  for  Networks 
and  Information  Integration/Department  of  Defense 
Chief  Information  Officer  and  provide  specificity  as 
to  when  and  where  these  functions  will  be 
transferred. 


In  October  201 1 ,  the  department  formally  disestablished  the 
Business  Transformation  Agency,  completing  the  transfer  of 
its  various  responsibilities  to  other  DOD  entities,  including  the 
Office  of  the  Deputy  Chief  Management  Officer.  In  addition, 
in  January  2012,  DOD  announced  the  disestablishment  of 
the  Assistant  Secretary  of  Defense  for  Networks  and 
Information  Integration  and  the  transfer  of  its  various 
responsibilities  to  other  DOD  entities,  including  the  DOD 
Chief  Information  Officer  and  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics. 


GAO-1 1-902:  Organizational  Transformation: 

Military  Departments  Can  Improve  Their 

Enterprise  Architecture  Programs,  Sep  26, 

2011a 

1.  To  ensure  that  the  military  departments  establish 
commitments  to  fully  develop  and  effectively 
manage  their  enterprise  architectures,  the 

Secretaries  of  the  Air  Force,  Army,  and  Navy  each 
should  expeditiously  provide  to  the  congressional 
defense  committees  a  plan  that  identifies 
milestones  for  their  respective  department’s  full 
satisfaction  of  all  of  our  Enterprise  Architecture 

X 

While  Department  of  the  Army  officials  stated  that  the 
department  plans  to  address  all  of  the  framework  elements, 
the  department  has  yet  to  develop  a  plan  for  doing  so. 
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In 

Yes  Process 

GAO  assessment 

Management  Maturity  Framework  elements.  In  the 
event  that  a  military  department  does  not  intend  to 
fully  satisfy  all  elements  of  our  framework,  the  plan 
should  include  a  rationale  for  why  the  department 
deems  any  such  element(s)  to  be  not  applicable. 

(ARMY) 

2.  To  ensure  that  the  military  departments  establish 
commitments  to  fully  develop  and  effectively 
manage  their  enterprise  architectures,  the 
Secretaries  of  the  Air  Force,  Army,  and  Navy  each 
should  expeditiously  provide  to  the  congressional 
defense  committees  a  plan  that  identifies 
milestones  for  their  respective  department’s  full 
satisfaction  of  all  of  our  Enterprise  Architecture 
Management  Maturity  Framework  elements.  In  the 
event  that  a  military  department  does  not  intend  to 
fully  satisfy  all  elements  of  our  framework,  the  plan 
should  include  a  rationale  for  why  the  department 
deems  any  such  element(s)  to  be  not  applicable. 
(NAVY) 

X 

The  Department  of  the  Navy  provided  a  memorandum 
describing  its  proposed  approach  for  addressing  the 
elements  of  the  framework  that  it  considers  applicable,  but 
this  memorandum  does  not  describe  the  department’s 
milestones  for  addressing  the  elements  and  the  department 
did  not  provide  evidence  to  support  that  the  memorandum 
has  been  delivered  to  the  congressional  defense  committees. 

3.  To  ensure  that  the  military  departments  establish 
commitments  to  fully  develop  and  effectively 
manage  their  enterprise  architectures,  the 
Secretaries  of  the  Air  Force,  Army,  and  Navy  each 
should  expeditiously  provide  to  the  congressional 
defense  committees  a  plan  that  identifies 
milestones  for  their  respective  department’s  full 
satisfaction  of  all  of  our  Enterprise  Architecture 
Management  Maturity  Framework  elements.  In  the 
event  that  a  military  department  does  not  intend  to 
fully  satisfy  all  elements  of  our  framework,  the  plan 
should  include  a  rationale  for  why  the  department 
deems  any  such  element(s)  to  be  not  applicable. 
(AIR  FORCE) 

X 

DOD  officials  reported  that  the  Air  Force  Enterprise 

Architecture  Management  Plan,  scheduled  for  issuance  in 

July  2013,  will  identify  milestones  for  satisfying  a  number  of 
the  elements  described  in  our  Enterprise  Architecture 
Management  Maturity  Framework. 

GAO-12-241:  Information  Technology: 

Departments  of  Defense  and  Energy  Need  to 

Address  Potentially  Duplicative  Investments, 

Feb  17,  2012 

1.  To  better  ensure  agencies  avoid  investing  in 
duplicative  investments,  the  Secretary  of  Defense 
should  direct  the  Chief  Information  Officer  to  correct 
the  miscategorizations  for  the  DOD  investments  we 
identified  and  ensure  that  investments  are  correctly 
categorized  in  agency  submissions. 

X 

DOD  has  taken  steps  to  help  ensure  accurate  categorizations 
of  its  IT  investments.  Specifically,  it  has  corrected 
miscategorized  investments  and  included  the  corrections  in 
its  budget  submissions. 

2.  To  better  ensure  agencies  avoid  investing  in 
duplicative  investments,  the  Secretary  of  Defense 
should  direct  the  Chief  Information  Officer  to  utilize 
existing  transparency  mechanisms,  such  as  the  IT 

X 

According  to  DOD  officials,  in  October  2012,  DOD  held  an  IT 
portfoliostat  review  in  which  it  evaluated  its  IT  investments  for 
fiscal  year  201 3.  In  addition,  as  discussed  in  this  report,  it 
also  began  to  implement  a  new  IRB  process  for  its  business 
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Implemented 

GAO  report  information  and  recommendation 

In 

Yes  Process 

GAO  assessment 

Dashboard,  to  report  on  the  results  of  the 
department’s  efforts  to  identify  and  eliminate,  where 
appropriate,  each  potentially  duplicative  investment 
we  have  identified,  as  well  as  any  other  duplicative 
investments. 

system  investments,  which  is  to,  among  other  things,  reduce 
duplication  and  improve  business  effectiveness.  However, 

DOD  has  not  yet  reported  on  whether  its  efforts  have  resulted 
in  the  identification  and  elimination  of  duplicative 
investments. 

GAO-12-685:  DOD  Business  Systems 
Modernization:  Governance  Mechanisms  for 
Implementing  Management  Controls  Need  to  Be 
Improved,  Jun  1,  2012 

1.  To  ensure  that  DOD  continues  to  implement  the  X 

full  range  of  institutional  management  controls 

needed  to  address  its  business  systems 
modernization  high-risk  area,  the  Secretary  of 
Defense  should  ensure  that  the  Deputy  Secretary 
of  Defense,  as  the  department’s  Chief  Management 
Officer,  establish  a  policy  that  clarifies  the  roles, 
responsibilities,  and  relationships  among  the  Chief 
Management  Officer,  Deputy  Chief  Management 
Officer,  DOD,  and  military  department  chief 
information  officers,  principal  staff  assistants, 
military  department  chief  management  officers,  and 
the  heads  of  the  military  departments  and  defense 
agencies,  associated  with  the  development  of  a 
federated  BEA.  Among  other  things,  the  policy 
should  address  the  development  and 
implementation  of  an  overarching  taxonomy  and 
associated  ontologies  to  help  ensure  that  each  of 
the  respective  portions  of  the  architecture  will  be 
properly  linked  and  aligned.  In  addition,  the  policy 
should  address  alignment  and  coordination  of 
business  process  areas,  military  department  and 
defense  agency  activities  associated  with 
developing  and  implementing  each  of  the  various 
components  of  the  BEA,  and  relationships  among 
these  entities. 

2.  To  ensure  that  annual  budget  submissions  are  X 

based  on  complete  and  accurate  information,  the 
Secretary  of  Defense  should  direct  the  appropriate 

DOD  organizations  to  establish  a  deadline  by  which 
it  intends  to  complete  the  integration  of  the 
repositories  and  validate  the  completeness  and 
reliability  of  information. 


3.  To  facilitate  congressional  oversight  and  promote  X 

departmental  accountability,  the  Secretary  of 
Defense  should  ensure  that  the  Deputy  Secretary 
of  Defense,  as  the  department’s  Chief  Management 
Officer,  direct  the  Deputy  Chief  Management 


The  department  stated  that  it  has  a  number  of  policy  and 
guidance  documents  currently  under  development  to  help 
clarify  the  roles  and  responsibilities  associated  with 
development  of  the  BEA  and  enterprise  architecture  in 
general  across  DOD.  For  example,  DOD  has  established  a 
forum  for  discussing  the  development  of  the  federated  BEA 
and  is  currently  working  to  establish  a  BEA  Configuration 
Control  Board  charter,  which  is  intended  to  document  the 
roles  and  responsibilities  for  the  review  and  approval  of 
proposed  BEA  content.  However,  DOD  has  yet  to  issue  these 
policy  and  guidance  documents. 


According  to  DOD,  the  department  is  fully  committed  to 
integrating  the  data  in  the  system  data  repositories.  DOD 
Chief  Information  Officer  officials  stated  that  a  new  portal  that 
will  provide  an  integrated  view  of  information  from  its  two 
portals  will  reach  a  limited  operational  capability  in  May  2013. 
However,  DOD  has  yet  to  establish  a  deadline  by  which  it 
intends  to  complete  the  integration  of  the  repositories  and 
validate  the  completeness  and  reliability  of  information. 

The  Office  of  the  Deputy  Chief  Management  Officer  stated 
that  measures  such  as  the  number  of  systems  that  have 
undergone  material  process  changes,  the  number  of 
interfaces  eliminated,  and  the  status  of  end-to-end  business 
process  reengineering  efforts  were  not  included  in  its  annual 
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Officer  to  include  in  DOD’s  annual  report  to 

Congress  on  compliance  with  10  U.S.C.  §  2222, 
the  results  of  the  department’s  business  process 
reengineering  efforts.  Among  other  things,  the 
results  should  include  the  department’s 
determination  of  the  number  of  systems  that  have 
undergone  material  process  changes,  the  number 
of  interfaces  eliminated  as  part  of  these  efforts  (i.e., 
by  program,  by  name),  and  the  status  of  its  end-to- 
end  business  process  reengineering  efforts. 

report  to  Congress  because  the  focus  for  this  year  was  on 
establishing  baseline  portfolios.  The  department  took  steps  to 
update  its  business  process  reengineering  guidance  in 
September  2012  and  has  begun  to  validate  business  process 
reengineering  assertions  on  selected  investments.  However, 
the  department  has  yet  to  identify  and  report  the  results  of 
the  department’s  business  process  reengineering  efforts. 

4.  To  facilitate  congressional  oversight  and  promote 
departmental  accountability,  the  Secretary  of 
Defense  should  ensure  that  the  Deputy  Secretary 
of  Defense,  as  the  department’s  Chief  Management 
Officer,  direct  the  Deputy  Chief  Management 

Officer  to  include  in  DOD’s  annual  report  to 

Congress  on  compliance  with  10  U.S.C.  §  2222,  an 
update  on  the  Office  of  the  Deputy  Chief 
Management  Officer’s  progress  toward  filling  staff 
positions  and  the  impact  of  any  unfilled  positions  on 
the  ability  of  the  office  to  conduct  its  work. 

X 

The  Office  of  the  Deputy  Chief  Management  Officer  provided 
an  update  to  GAO  on  the  numbers  of  positions  filled  and 
open.  However,  status  on  staffing  and  the  impact  of  unfilled 
positions  on  the  ability  of  the  office  to  conduct  its  work  have 
not  yet  been  included  in  the  annual  report  to  Congress. 

GAO-12-791:  Organizational  Transformation: 

Enterprise  Architecture  Value  Needs  to  Be 

Measured  and  Reported ,  Sep  26,  2012 

1.  To  enhance  federal  agencies’  ability  to  realize 
enterprise  architecture  benefits,  the  Secretary  of 
Defense  should  periodically  measure  and  report 
enterprise  architecture  outcomes  and  benefits  to 
top  agency  officials  (i.e.,  executives  with  authority 
to  commit  resources  or  make  changes  to  the 
program)  and  to  the  Office  of  Management  and 
Budget  (OMB). 

X 

According  to  DOD,  the  Defense  Business  Council  is  currently 
chartering  the  BEA  Configuration  Control  Board  and  will 
require  all  BEA  changes  to  be  directly  traceable  to  a 
measureable  business  outcome.  However,  the  department 
has  yet  to  measure  and  report  enterprise  architecture 
outcomes  and  benefits  consistent  with  our  recommendation 
or  report  the  results  of  outcome  measurements  to  OMB.  In 
addition,  the  department  did  not  demonstrate  that  its  military 
departments  or  its  corporate  enterprise  architecture  efforts 
are  periodically  measuring  and  reporting  related  outcomes 
and  benefits  to  top  agency  officials  and  to  OMB.  OMB  has 
requested  agencies  to  report  enterprise  architecture 
outcomes  in  reports  that  are  due  by  May  15,  2013. 

Source:  GAO  analysis  of  DOD  and  OMB  information. 


aGAO-1 1-902  focused  on  the  military  departments’  corporate  architectures. 
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DEPUTY  CHIEF  MANAGEMENT  OFFICER 
9010  DEFENSE  PENTAGON 
WASHINGTON,  DC  20301-9010 


MAY  1  0  2013 


Ms.  Valerie  Melvin 

Director,  Information  Management  and  Technology  Resources  Issues 
U.S.  Government  Accountability  Office 
441  G  Street,  NW 
Washington,  DC  20548 

Dear  Ms.  Melvin, 

This  is  the  Department  of  Defense  (DoD)  response  to  the  Government  Accountability 
Office  (GAO)  draft  report  GAO- 13-557,  “DOD  BUSINESS  SYSTEMS  MODERNIZATION: 
Further  Actions  Needed  to  Address  Challenges  and  Improve  Accountability,"  dated  May  1, 

2013,  (GAO  Code  310993).  While  there  are  a  few  recommendations  with  which  the  Department 
concurs,  on  the  whole  we  disagree  with  many  of  the  assertions  put  forward  by  GAO  and  the  lack 
of  balance  expressed  in  the  draft  report. 

Every  day  the  Department  successfully  operates  one  of  the  most  complex  and  diverse 
business  enterprises  in  the  world,  and  DoD  leaders  routinely  take  deliberate  actions  to  improve 
how  the  Department  conducts  its  business.  The  Department  has  implemented  a  business 
management  model  that  improves  services  to  the  warfighter  by  aligning  the  Department’s 
business  strategies  and  desired  outcomes  with  information  technology  investments  organized  as 
portfolios.  The  draft  report  demonstrates  a  lack  of  understanding  of  this  new  investment 
management  process  and  the  significant  improvements  it  has  produced.  During  Fiscal  Year 
2012,  we  took  full  advantage  of  the  mandate  in  tide  10  United  States  Code  section  2222  to 
revamp  business  mission  area  governance  and  establish  an  Integrated  Business  Framework. 

DoD  business  leaders  now  use  this  framework  to  vet  business  strategy,  link  portfolio  objectives 
to  enterprise  vision,  mission,  goals  and  objectives,  and  ensure  business  process  reengineering 
and  business  enterprise  architecture  activides  are  aligned  to  end-to-end  processes  to  make 
informed  business  system  investment  and  certificadon  decisions.  Rather  than  the  slow 
improvement  portrayed  in  the  draft  report,  these  changes  have  resulted  in  significant  progress. 

While  our  business  systems  modernization  efforts  will  continue  to  evolve,  it  is  important 
for  the  Department  and  Congress  not  to  lose  sight  of  the  bigger  picture  of  what  we  endeavor  to 
achieve  by  focusing  only  on  the  set  of  challenges  cited  by  GAO  in  the  draft  report.  DoD’s 
Business  Mission  Area  is  a  complex,  interconnected  environment  that  is  unprecedented  in  size 
and  critical  to  the  Department’s  overall  national  security  mission.  Rather  than  attempt  to 
institute  a  one-size-fits-all  approach  across  every  aspect  of  the  DoD  business  enterprise,  the 
Department  will  continue  to  prioritize  actions  and  activities  that  produce  the  best  return  on 
investment.  Implementing  GAO’s  recommendations  to  institute  a  100-percent  compliance 
verification  regime  without  regard  to  the  lifecycle  phase  of  a  program  would  divert  scarce 
resources  for  tasks  that  provide  no  benefit,  for  example,  reengineering  a  system  targeted  to  be 
retired.  To  separately  track  milestone  and  performance  data  for  investments  as  part  of  the 
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investment  review  process,  as  GAO  also  suggests,  would  be  duplicative  of  the  Department’s 
robust,  existing  controls  that  are  part  of  our  Defense  Acquisition  System. 

Finally,  there  is  significant  focus  in  the  report  on  the  Department’s  failure  to  implement 
recommendations  that  are  almost  a  decade  old  and  that  have  been  overtaken  by  more  recent 
events.  We  have  made  progress  over  the  years  in  implementing  those  recommendations  with 
which  we  concurred;  however,  the  Department’s  business  modernization  efforts  have  evolved 
significantly  since  2005.  In  March,  the  Department  issued  revised  guidance  for  FY2014  funds 
certification  that  incorporates  lessons  learned  during  our  first  year  using  the  new  process.  Going 
forward,  we  will  continue  to  mature  and  refine  our  efforts  to  rationalize  the  Department’s 
business  system  investments  to  eliminate  unnecessary  redundant  systems,  as  well  as  identifying 
both  long  range  improvement  opportunities  and  near-term  steps  to  advance  current  business 
practices. 

The  Department  appreciates  the  opportunity  to  respond  to  your  draft  report.  We  look 
forward  to  your  continued  cooperation  and  dialog  toward  our  common  goal  of  improving 
business  systems  modernization  throughout  the  Department  of  Defense.  Should  you  have  any 
questions,  please  contact  Mr.  Thomas  Cowley,  703-692-8170,  thomas.cowley@osd.mil. 

Sincerely, 

Elizabeth  A.  McGrath 

Enclosures: 

As  stated 
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GAO  DRAFT  REPORT  DATED  APRIL  30, 2013 
GAO-13-S57  (GAO  CODE  310993) 

“DOD  BUSINESS  SYSTEMS  MODERNIZATION:  FURTHER  ACTIONS 
NEEDED  TO  ADDRESS  CHALLENGES  AND  IMPROVE  ACCOUNTABILITY” 

DEPARTMENT  OF  DEFENSE  COMMENTS 
TO  THE  GAO  RECOMMENDATIONS 


RECOMMENDATION  1:  The  Government  Accountability  Office  (GAO)  recommends 
that  the  Secretary  of  Defense  direct  the  Deputy  Chief  Management  Officer  (DCMO)  to 
define  by  when  and  how  the  department  plans  to  develop  an  architecture  that  would 
extend  to  all  defense  components  and  include,  among  other  things,  (a)  information  about 
the  specific  business  systems  that  support  Business  Enterprise  Architecture  (BEA) 
business  activities  and  related  system  functions;  (b)  business  capabilities  for  the  Hire-to- 
Retire  and  Procure-to-Pay  business  processes;  (c)  sufficient  information  about  business 
activities  to  allow  for  more  effective  identification  of  potential  overlap  and  duplication. 

DoD  RESPONSE:  Partially  concur.  The  Department  of  Defense  (DoD)  agrees  that 
plans  for  continuing  to  mature  the  Federated  BEA  should  be  completed.  It  should  be 
noted  that  the  Department  has  already  taken  the  necessary  initial  steps  to  extend  the  BEA 
to  all  Defense  Components.  These  steps  have  been  institutionalized  via  the  Integrated 
Business  Framework  which  was  implemented  during  calendar  year  2012.  The 
framework  links  together  the  development  of  functional  business  strategies,  the 
identification  of  associated  BEA  changes,  and  the  submission  of  Organizational 
Execution  Plans  (OEPs)  from  the  DoD  Components.  Information  contained  in  the 
Component-developed  OEPs  constitutes  key  elements  of  the  Component's  portion  of  the 
Federated  BEA.  In  particular,  the  OEPs  and  associated  authoritative  data  sources  provide 
the  relationship  and  linkage  between  each  Component's  portfolio  of  proposed 
investments  and  key  elements  of  the  corporate  portion  of  the  Federated  BEA. 


RECOMMENDATION  2:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Deputy  Chief  Management  Officer  to  define  by  when  and  how  the  enterprise 
transition  plan  will  include,  among  other  things,  (a)  milestones,  performance  measures, 
and  funding  plans  for  all  business  systems  expected  to  be  part  of  the  target  architecture 
and  each  system’s  risks  or  challenges  to  integration;  (b)  time-phased  end  dates  associated 
with  terminating  legacy  systems  in  phases;  (c)  a  listing  of  legacy  systems  that  will  be  a 
part  of  the  target  defense  business  systems  computing  environment  and  a  strategy  for 
making  modifications  to  legacy  systems  that  will  be  needed  to  ensure  that  they  comply 
with  the  defense  BEA,  including  time-phased  milestones,  performance  measures,  and 
financial  resource  needs  and  (d)  information  about  how  systems  are  to  be  sequenced 
according  to,  among  other  things,  dependencies  among  investments. 


Page  75 


GAO-13-557  DOD  Business  Systems  Modernization 


Appendix  IV:  Comments  from  the  Department 
of  Defense 


DoD  RESPONSE:  Partially  concur.  The  Department  is  meeting  the  requirements  of 
section  2222  as  follows:  (a)  milestones,  performance  measures,  and  funding  plans  for  all 
business  systems  expected  to  be  part  of  the  target  architecture  and  each  system’s  risks  or 
challenges  to  integration.  This  information  is  explained  in  the  presentations  that  are 
provided  to  the  Defense  Business  Council  (DBC);  (b)  time-phased  end  dates  associated 
with  terminating  legacy  systems  in  phases.  This  requirement  is  being  met  by  the 
establishment  of  a  category  of  “legacy”  systems  that  are  to  sunset  within  the  next  36 
months  along  with  an  explanation  of  what  if  any  systems  will  continue  the  business  need; 
(c)  a  listing  of  systems  that  will  be  a  part  of  the  target  defense  business  systems 
computing  environment  and  a  strategy  for  making  modifications  to  systems  that  will  be 
needed  to  ensure  that  they  comply  with  the  Defense  BEA,  including  time-phased 
milestones,  performance  measures,  and  financial  resource  needs.  This  requirement  is 
being  met  by  the  precertification  process  and  through  the  Defense  acquisition  system 
milestones,  performance  measures,  etc.;  d)  Information  about  how  systems  are  to  be 
sequenced  according  to,  among  other  things,  dependencies  among  investments.  This 
information  is  captured  in  presentations  and  supporting  materials  provided  to  the  DBC. 

The  Department  is  using  core  attributes  of  the  GAO  Information  Technology  Investment 
Management  (ITIM)  framework  and  the  Capital  Planning  and  Investment  Control  (CPIC) 
process  to  manage  DoD’s  budget  and  acquisition  of  defense  business  systems.  DoD  has 
developed  a  complete  investment  portfolio,  established  selection  criteria,  and  developed 
an  Integrated  Business  Framework  that  continues  to  evolve  for  selecting,  controlling  and 
evaluating  investments.  The  Department  built  its  investment  foundation  by  establishing 
organizational  priorities  through  the  Strategic  Management  Plan  and  aligning  Functional 
Strategies  that  are  then  executed  by  DoD  Components  through  their  OEPs.  DoD  is 
focusing  on  delivering  efficient,  effective,  and  agile  business  operations  that  support  and 
enable  the  warfighter;  achieving  specific  program  business  outcomes;  using  criteria 
related  to  return  on  investment  and  quantitative  and  qualitative  criteria  for  comparing  and 
prioritizing  alternative  defense  business  systems  programs;  realizing  enterprise 
architecture-driven  performance  improvements  and  outcomes  (e.g.,  improving  mission 
performance;  saving  money  and  avoiding  costs;  enhancing  the  quality  of  DoD  investment 
portfolios;  improving  the  quality,  availability,  and  sharing  of  data  and  information;  and 
increasing  the  transparency  of  DoD  operations)  to  complement  existing  and  enduring 
budget  and  acquisition  processes  within  DoD. 

The  Defense  Acquisition  System  is  the  core  process  used  by  the  Department  to  manage 
defense  acquisitions  to  include  the  procurement  of  defense  business  systems.  Likewise, 
the  Department’s  Planning,  Programming,  Budgeting  and  Execution  (PPBE)  process  is 
the  principal  process  for  managing  the  DoD  budget.  Taken  together,  the  Department  has 
a  much  greater  visibility  into  the  business  enterprise  than  ever  before.  The  Department  is 
taking  a  data-driven  approach  to  manage  defense  business  systems  as  portfolios  of 
investments  within  the  Integrated  Business  Framework.  It  is  our  goal  to  leverage  and 
aggregate  data  from  authoritative  data  sources  and  tools  used  by  the  PPBE,  acquisition 
and  funds  certification  processes  to  track  and  manage  the  overall  performance  of 
portfolios. 
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The  Department  considers  any  Defense  Business  System  (DBS)  with  a  sunset  date  within 
36  months  of  certification  as  legacy  and  development  and  modernization  funds  for  these 
legacy  systems  have  been  removed  from  the  budget.  All  other  defense  business  systems 
are  considered  core  and  therefore  must  demonstrate  compliance  to  the  BEA  while 
ensuring  appropriate  Business  Process  Re-Engineering  (BPR).  Modernization  to 
individual  systems  will  be  tracked  and  monitored  by  the  Defense  Acquisition  System; 
however,  the  requirement  driving  the  modernization  will  be  validated  by  the  Defense 
Business  Council  (DBC)  in  accordance  with  Chairman  of  the  Joint  Chiefs  of  Staff 
Instruction  3170.01H,  “Joint  Capabilities  Integration  and  Development  System,”  and 
Directive-Type  Memorandum  (DTM)  11-009,  “Acquisition  Policy  for  Defense  Business 
Systems,”  to  ensure  strategic  alignment  with  the  Strategic  Management  Plan,  Functional 
Strategies  and  the  BEA.  Further,  the  DBC  will  continue  to  track  the  performance  of 
business  system  portfolios  as  OEPs  are  presented  to  the  DBC  for  certification. 


RECOMMENDATION  3:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Deputy  Chief  Management  Officer  to  ensure  that  the  functional  strategies  include  all 
of  the  critical  elements  identified  in  DOD  investment  management  guidance,  including 
performance  measures  to  determine  progress  toward  achieving  the  goals  that  incorporate 
all  of  the  attributes  called  for  in  the  department’s  guidance. 

DoD  RESPONSE:  Concur. 


RECOMMENDATION  4:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Deputy  Chief  Management  Officer  to  select  and  control  its  mix  of  investments  in  a 
manner  that  best  supports  mission  needs  by  (a)  documenting  a  process  for  evaluating 
portfolio  performance  that  includes  the  use  of  actual  versus  expected  performance  data 
and  predetermined  thresholds;  (b)  ensuring  that  portfolio  assessments  are  conducted  in 
key  areas  identified  in  our  IT  investment  management  framework:  benefits  attained; 
current  schedule;  accuracy  of  project  reporting;  and  risks  that  have  been  mitigated, 
eliminated,  or  accepted  to  date;  and  (c)  ensuring  that  the  documents  provided  to  the 
Defense  Business  Council  as  part  of  the  investment  management  process  include  critical 
information  for  conducting  all  assessments. 

DoD  RESPONSE:  Non-concur.  This  recommendation  does  not  recognize  the 
significant  improvements  the  Department  made  in  2012  to  fundamentally  change  how  it 
reviews  its  defense  business  systems.  The  FY2012  NDAA  substantially  changed  the 
scope  and  governance  approach  for  reviewing  and  certifying  defense  business  systems  by 
creating  a  single  Investment  Review  Board  (IRB)  to  review  all  business  systems  planning 
to  spend  more  than  $1M  over  the  period  of  the  Future  Years  Defense  Program  (FYDP) 
regardless  of  the  funding  source.  In  response  to  the  NDAA,  the  Department  created  the 
Integrated  Business  Framework,  a  paradigm  shift  that  aligns  business  strategies  to 
business  systems  IT  spending  vice  simply  looking  at  IT  costs. 
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Regarding  the  specific  recommendations: 

(a)  Process  Documentation:  The  Department  created  initial  documentation 
aligned  to  the  Integrated  Business  Framework  in  June  2012.  Section  3.3  of 
the  Defense  Business  System  Investment  Review  Guidance  defines  the  review 
criteria  used  by  the  DBC  to  evaluate  and  certify  business  systems  via  OEPs 
for  certification.  Based  on  lessons  learned  in  2012,  we  then  updated  guidance 
to  further  strengthen  and  improve  the  process.  As  noted  in  the  draft  report,  the 
Functional  Strategies  that  were  updated  in  2013  include  improved  business 
goals  and  supporting  measures  that  will  be  used  to  evaluate  business  system 
certification  requests  when  they  are  presented  to  the  DBC  later  this  year.  It  is 
our  intent,  unless  a  business  system  is  required  for  critical  operational  support, 
to  withhold  certification  for  business  system  investments  that  are  not  aligned 
to  the  Functional  Strategies.  Likewise,  it  is  not  the  intent  of  the  DBC  (which 
serves  as  the  Department’s  IRB)  to  review  acquisition  issues  associated  with 
individual  business  systems,  as  that  function  is  performed  by  the  Defense 
Acquisition  System. 

(b)  Portfolio  assessments  in  accordance  with  GAO  IT  investment 
management  framework:  The  Department  is  leveraging  key  components  of 
the  GAO  ITIM  Framework  to  evaluate  portfolios  of  investments  presented  to 
the  DBC.  However  the  IRB  process  will  complement  the  Department’s 
acquisition  system  rather  than  duplicate  it.  The  Defense  Acquisition  System 
is  the  core  process  used  by  the  Department  to  manage  defense  acquisitions,  to 
include  the  procurement  of  defense  business  systems,  and  it  already  contains 
significant  elements  of  the  GAO  ITIM  framework. 

(c)  DBC  documentation  for  conducting  assessments:  This  aspect  of  the 
recommendation  does  not  take  into  account  that  the  information  derived  for 
strategic  alignment,  cost,  utility  and  compliance  is  already  given  to  the  DBC 
and  incorporates  objective  information  derived  from  authoritative  documents 
and  data  sources.  Additionally,  the  documentation  is  supported  by  input  from 
the  Functional  Area  Owners  and  Subject  Matter  Experts. 


RECOMMENDATION  5:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Deputy  Chief  Management  Officer  to  implement  and  use  the  BEA  and  business 
process  reengineering  compliance  assessments  more  effectively  to  support  organizational 
transformation  efforts  by  (a)  disclosing  relevant  information  about  known  weaknesses, 
such  as  BEA  and  business  process  reengineering  compliance  weaknesses  for  systems  that 
were  not  certified  or  certified  with  qualifications  in  annual  reports  to  Congress;  (b) 
establishing  milestones  by  which  selected  validations  of  BEA  compliance  assertions  are 
to  be  completed;  and  (c)  ensuring  that  appropriate  business  process  reengineering 
assertions  have  been  completed  on  all  investments  submitted  for  the  fiscal  year  2014 
certification  reviews  prior  to  the  certification  of  funds. 
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DoD  RESPONSE:  Partially  concur.  DoD  agrees  that  BEA  and  BPR  compliance 
assessments  be  used  to  support  organizational  transformation  efforts  by  disclosing 
relevant  information  about  known  weaknesses  and  that  milestones  should  be  identified 
for  completing  selected  validations.  However,  DoD  does  not  agree  with  the  GAO 
recommendation  to  ensure  that  appropriate  business  process  reengineering  assertions  be 
completed  on  aU  investments  submitted  for  the  fiscal  year  2014  certification  reviews 
prior  to  the  certification  of  funds.  Requiring  BPR  for  systems  scheduled  for  retirement  or 
for  systems  solely  in  sustainment  would  divert  scarce  resources  for  tasks  that  would 
provide  no  benefit.  Therefore  our  transition  plan  for  those  systems  is  to  sunset  the 
program  within  36  months  (legacy)  and,  for  systems  solely  in  sustainment,  determine  that 
that  it  is  not  appropriate  to  conduct  further  BPR.  The  revised  BPR  Assessment  Guidance 
and  memorandum  of  September  28, 2012  documents  the  DCMO  BPR  Assessment  plan 
which  includes  providing  the  Department  a  BPR  Standard,  outlining  a  plan  of  action  for 
executing  a  validation  of  a  sampling  of  DBS  programs  in  which  the  Pre-Certification 
Authority  (PCA)  asserted  BPR  compliance  on  their  FY2013  Organizational  Execution 
Plan  (OEP).  The  revised  BPR  Assessment  Guidance  memorandum  also  required  the 
PCA  to  develop  a  BPR  Plan  of  Action  for  all  DBS  programs  which  did  not  assert  BPR 
compliant  in  FY2013  to  become  BPR  compliant;  this  requirement  was  also  documented 
in  the  Investment  Decision  Memorandum  (IDM)  for  each  OEP  as  a  condition  to  the 
FY2013  certification  of  funds.  The  results  of  the  Office  of  the  DCMO  BPR  Assessment 
validations  and  the  PCA  response  to  the  IDM/BPR  Compliance  Plan  of  Action  will  be 
reported  to  the  DBC  this  summer. 

The  DoD  is  an  operational  organization  with  emerging  and  unforeseen  operational  needs, 
therefore  the  DBC  must  retain  and  use,  when  appropriate,  the  conditional  certification 
authority  as  it  relates  to  the  BPR  compliance  requirement.  This  authority  has  been  used 
by  the  DoD  IRBs  since  the  inception  of  the  certification  review  process  in  2005  and 
allows  the  DBC  to  levy  a  condition  on  the  DBS  program  to  develop  a  BPR  Plan  of 
Action  by  a  specified  date.  It  is  the  expectation  of  the  DBC  that  all  BPR  compliance 
assertions  be  true  and  verifiable;  having  the  ability  to  condition  the  DBS  program  with  a 
BPR  Plan  of  Action  supports  the  DBC’s  ability  to  go  back  and  potentially  assess  the  BPR 
effort  based  on  their  BPR  Plan  of  Action. 


RECOMMENDATION  6:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  Deputy  Chief  Management  Officer  to  develop  a  skills  inventory,  needs  assessment, 
gap  analysis,  and  plan  to  address  identified  gaps  as  part  of  a  strategic  approach  to  human 
capital  planning. 

DoD  RESPONSE:  Non-concur.  The  Office  of  the  DCMO  is  a  relatively  small 
component  of  the  Office  of  the  Secretary  of  Defense.  The  proposed  direction  could  not 
be  accomplished  given  current  resources  (both  personnel  and  funding)  and  is  a  level  of 
activity  normally  designed  for  an  entire  agency.  The  draft  report  also  draws  conclusions 
from  statements  by  Office  of  the  DCMO  officials  as  to  the  impact  of  the  Office  of  the 
DCMO’s  staffing  levels  that  are  not  supported  by  the  findings.  The  Office  of  the  DCMO 
is  constantly  working  to  improve  its  human  capital  assets  in  ways  that  are  appropriate  to 
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an  office  with  less  than  100  government  employees.  As  part  of  the  functional  work 
assessment  and  reorganization  GAO  referenced  in  the  report,  the  Office  of  the  DCMO 
thoroughly  considered  the  skills  of  its  employees,  sought  to  optimally  align  them 
organizationally,  and  has  used  hiring  actions  to  bring  additional  skill  sets  onboard  that 
filled  identified  gaps.  The  report  attributes  consequences  of  the  Office  of  the  DCMO’s 
perceived  human  capital  weaknesses  far  beyond  what  is  supported  by  the  GAO’s 
findings.  While  the  Office  of  the  DCMO  sets  policy,  performs  oversight  activities,  and 
develops  important  products,  it  relies  on  content,  input,  and  implementation  from 
throughout  the  DoD  Components  to  move  forward.  Weaknesses  identified  by  GAO  in 
the  Department’s  federated  architecture,  investment  management  data,  and  transition  plan 
content  are  Departmental  weaknesses  and  are  not  simply  due  to  the  staffing  levels  of  the 
Office  of  the  DCMO. 


RECOMMENDATION  7:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  appropriate  authority  to  ensure  that  complete  documentation,  such  as  root  cause 
analyses,  assessments  of  existing  interfaces  for  reuse  opportunities,  and  performance 
metrics  related  to  the  reengineering  efforts  is  provided  as  part  of  the  fiscal  year  2014 
certification  and  approval  process  for  the  IPPS-A,  IPPS-N,  AF-IPPS,  and  iEHR 
investments. 

DoD  RESPONSE:  Concur.  The  revised  BPR  Assessment  Guidance  of  September  28, 
2012  specifically  addressed  root  cause  analyses  in  question  5,  assessment  of  interfaces  in 
question  10,  and  performance  measures  in  questions  4  and  1 1  with  the  expectation  that 
objective  evidence  is  available  that  supports  the  responses  to  questions  4,  5, 10,  and  1 1. 

It  is  the  expectation  of  the  DBC  that  all  PCA  BPR  compliance  assertions  are  made  using 
the  BPR  Assessment  Form  with  objective  evidence  which  should  yield  a  more  accurate 
evaluation  and  provide  a  more  tangible  outcome  for  all  DBS  programs. 


RECOMMENDATION  8:  The  GAO  recommends  that  the  Secretary  of  Defense  direct 
the  appropriate  authority  to  determine  whether  funds  were  properly  obligated  under  10 
U.S.C.  §2222(a)-(b)  for  systems  for  which  appropriate  business  process  reengineering 
assertions  were  not  completed. 

DoD  RESPONSE:  Non-concur.  The  Department  believes  that  it  has  already  met  the 
conditions  of  10  U.S.C.  §2222  through  reviews  by  the  Pre-Certification  authorities  and 
the  DBC.  As  an  example,  the  Department  considers  any  Defense  Business  System 
(DBS)  with  a  sunset  date  within  36  months  of  certification  as  legacy  so  development  and 
modernization  funds  for  these  legacy  systems  have  been  removed  from  the  budget.  The 
transition  plan  for  these  legacy  systems  is  to  sunset  the  system  as  opposed  to  bringing  the 
system  into  compliance  with  the  BEA.  This  action  is  in  the  best  interest  of  the 
department  and  conserves  resources.  The  requirement  to  sunset  the  DBS  is  included  in 
all  relevant  documentation  generated,  is  approved  by  the  DBC  and  the  DBSMC  and 
therefore  has  been  reviewed  for  compliance. 
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All  other  defense  business  systems  are  considered  core  and  therefore  must  demonstrate 
compliance  or  a  transition  plan  for  compliance  to  the  BEA  while  ensuring  appropriate 
Business  Process  Re-Engineering  (BPR).  Modernization  to  individual  systems  will  be 
tracked  and  monitored  by  the  Defense  Acquisition  System;  however,  the  requirement 
driving  the  modernization  will  be  validated  by  the  Defense  Business  Council  (DBC)  in 
accordance  with  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  3 170.01  H,  “Joint 
Capabilities  Integration  and  Development  System,”  and  Directive-Type  Memorandum 
(DIM)  11-009,  “Acquisition  Policy  for  Defense  Business  Systems,”  to  ensure  strategic 
alignment  with  the  Strategic  Management  Plan,  Functional  Strategies  and  the  BEA. 
Further,  the  DBC  will  continue  to  track  the  performance  of  business  system  portfolios  as 
OEPs  are  presented  to  the  DBC  for  certification. 

Further,  to  strengthen  BPR  assertions,  the  Department  issued  the  revised  BPR 
Assessment  Guidance  memorandum  of  September  28, 2012,  which  specifically  addressed 
the  requirement  for  the  PCA  to  develop  a  BPR  Plan  of  Action  for  all  DBS  programs  that 
did  not  assert  BPR  compliance  in  FY2013  to  become  BPR  compliant;  this  requirement 
was  also  documented  in  the  IDM  for  each  OEP  as  a  condition  to  the  FY2013  certification 
of  funds.  The  results  of  the  Office  of  the  DCMO  BPR  Assessment  validations  and  the 
PCA  response  to  the  IDM/BPR  compliance  Plan  of  Action  will  be  reported  to  the  DBC 
this  summer.  It  is  the  expectation  of  the  DBC  that  a  larger  number  of  DBS  programs  will 
assert  BPR  compliance  in  FY2014  (prior  to  the  fund  certification  requests,  in  either  an 
OEP  or  out  of  cycle).  As  noted,  the  DBC  does  not  require  BPR  for  systems  scheduled  for 
retirement.  For  systems  solely  in  sustainment,  a  determination  is  made  that  further  BPR  is 
not  required  as  it  would  divert  scarce  resources  for  tasks  that  would  provide  no  benefit. 
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